Parrot 4.5.0 "Buff-faced Pygmy Parrot" | parrot.org/ | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 29 June 2012.
Coke chromatic++ 01:55
pmichaud: wow, that link is subtle. 01:58
Danke.
dukeleto chromatic++ lives! 02:16
04:31 alvis_ joined 04:49 fperrad joined 05:04 alvis_ joined
moritz msg whiteknight: I still get the "Lossy conversion to single byte encoding" on your branch in rakudo's socket tests 05:17
aloha OK. I'll deliver the message.
05:39 Khisanth joined 06:51 brrt joined 07:00 he joined
dalek kudo/nom: cef4452 | moritz++ | src/core/terms.pm:
do not die when assigning non-Str values to %*ENV<..> elements
 07:09
07:38 nopaste joined 07:45 dalek joined 07:50 he joined 08:07 lucian joined 09:20 schmooster joined 09:23 brambles joined
dalek kudo/nom: ee8ca1a | moritz++ | src/core/Pod.pm:
use fully qualified names for Pod classes

that way the .perl output can be eval()ed again		 09:47
09:50 dukeleto joined
dalek p/toqast: 8a787d6 | moritz++ | VERSION:
bump VERSION
 10:00
p/toqast: c7f76bc | moritz++ | VERSION:
Merge branch 'release-2012.06.1'
p/toqast: 5a8707b | pmichaud++ | src/HLL/Compiler.pm:
Add -V/--verbose-config option to HLL::Compiler (RT #62462).
p/toqast: 7d3441a | pmichaud++ | src/HLL/Compiler.pm:
Implement --stagestats=2; outputs memory and PMC allocation statistics
p/toqast: bec0eba | pmichaud++ | src/HLL/Compiler.pm:
Restore a basic dumping capability from Parrot's Data::Dumper.
p/toqast: 2bd8dcb | pmichaud++ | src/HLL/Compiler.pm:
HLL::Compiler --stagestats=3 (or greater) now introduces prompts after each compilation stage.

check for system resource usage.
p/toqast: f16f2c5 | jnthn++ | src/how/NQPModuleHOW.pm:
Stop nqp::can(EXPORTHOW, 'foo') from exploding by adding some missing bits to NQPModuleHOW.
p/toqast: 095a387 | jnthn++ | / (3 files):
Merge branch 'master' into toqast
p/toqast: e262e92 | jnthn++ | src/QAST/Compiler.nqp:
A bit more diagnostic output.
 10:14
p/toqast: 52ba417 | jnthn++ | / (3 files):
Get first pieces of QAST::Want support in place.
kudo/toqast: 917c855 | jnthn++ | src/QPerl6/Compiler.nqp:
Disable optimizer for now; will update it for QAST later on.
 10:17
kudo/toqast: 471bd15 | jnthn++ | src/QPerl6/ (2 files):
Start translating a few lexicals bits to QAST.
kudo/toqast: 519c853 | jnthn++ | src/QPerl6/World.pm:
Translate various fixup code.
kudo/toqast: c3cfc12 | jnthn++ | src/QPerl6/ (2 files):
Various Op and Var translations to try and get us towards compiling hello world.
kudo/toqast: 71518ca | jnthn++ | src/QPerl6/World.pm:
Fix setting loading code.
kudo/toqast: 86ae30b | jnthn++ | src/QPerl6/World.pm:
PAST::Want -> QAST::Want in World.
kudo/toqast: d64eedd | jnthn++ | src/QPerl6/Actions.pm:
Disable context saving for a bit (needs updating for QAST).
10:19 Psyche^ joined 10:29 whiteknight joined
whiteknight good morning, #parrot 10:40
brrt good afternoon 10:42
whiteknight hello brrt 10:47
brrt i thought some more about it 10:48
i don´t think you need to use a linked list
(for api_jmp_buf)
is it used outside of the parrot api?
you can just keep its ´old´ value on the stack 10:50
whiteknight no, it is only used in the api
I can definitely do that. The API is still not intended to be used through NCI. I'm positive there's a better way 10:51
brrt :-) 10:52
the other way simply consists of me extending parrot to make a pmc extending apr_table_t and apr_array_header_t
whiteknight well, you can still use NCI, just with some functions better designed for it 10:53
The embedding API is intended for use by code outside Parrot. From the runloop you're very much inside it 10:54
brrt the point is i want to make some objects that are ´useful´ and contain parameters from apache 10:56
headers, request parameters
whiteknight right
brrt the first one is in a (hash) table structure
the second one in a specific C structure
i´d like to extract them ´on demand´
whiteknight right 10:57
StructView or a custom PMC type may be the best options 10:58
brrt i guess that is so
dalek p/toqast: 4621c27 | jnthn++ | src/QAST/Compiler.nqp:
Fix loadlibs compilation.
 11:23
p/toqast: ef8c730 | jnthn++ | src/QAST/Compiler.nqp:
Add stub for where we'll generate deserialization code later.
 11:24
p/toqast: 2a0ef4c | jnthn++ | src/QAST/Compiler.nqp:
No blocktype implies declaration.
kudo/toqast: 3e204d4 | jnthn++ | src/QPerl6/Actions.pm:
Updating various other PAST::Want usages to QAST::Want.
 11:25
11:45 JimmyZ joined 11:50 JimmyZ joined
dalek kudo/toqast: 4fc6c4b | jnthn++ | src/QPerl6/ (2 files):
Update a bunch of PAST::Var lexical lookups to use QAST::Var.
 12:30
kudo/toqast: a4e92a9 | jnthn++ | src/QPerl6/World.pm:
Fix a get_slot_past_for_object leftover.
kudo/toqast: 3ca0be7 | jnthn++ | src/QPerl6/Actions.pm:
Start updating EXPR action method for QAST.
kudo/toqast: 65346e6 | jnthn++ | src/QPerl6/Actions.pm:
Disable whatever-currying code for now; will need quite some updating.
kudo/toqast: 611bfb9 | jnthn++ | src/QPerl6/Actions.pm:
.type => .returns
 12:39
12:45 PacoAir joined 12:54 dukeleto joined 13:00 dngor_ joined 13:07 JimmyZ_ joined 13:16 mtk joined 13:21 JimmyZ joined 13:38 JimmyZ_ joined 13:42 bluescreen joined
dalek p/toqast: 325137a | jnthn++ | src/QAST/Compiler.nqp:
Ensure the QAST compiler is registered and the operations manager is available.
 14:09
p/toqast: f51d4a9 | jnthn++ | src/QAST/Compiler.nqp:
Honor HLL in operation lookups.
 14:40
kudo/toqast: 8f86af5 | jnthn++ | src/QPerl6/Actions.pm:
nsentry is dead
 14:47
kudo/toqast: eaff765 | jnthn++ | / (3 files):
Move Perl 6 ops registration out into a different file, and update these to register the ops with QAST.
kudo/toqast: db3f53b | jnthn++ | src/QPerl6/ (3 files):
Add a nqp::op for perl6_container_store and update actions to use it.
kudo/toqast: 673fc87 | jnthn++ | src/QPerl6/Ops.pm:
Fix some bogus whitespace.
kudo/toqast: 567eb74 | jnthn++ | src/QPerl6/Actions.pm:
perl6_booleanize => p6bool
kudo/toqast: ccd30a7 | jnthn++ | src/QPerl6/ (2 files):
perl6ize_type => p6type
kudo/toqast: 5761ab7 | jnthn++ | src/QPerl6/ (2 files):
perl6_take_dispatcher => p6takedisp
kudo/toqast: 3e84f10 | jnthn++ | src/QPerl6/Actions.pm:
Look for QAST::Node instead of PAST::Node.
 15:11
kudo/toqast: 71fb485 | jnthn++ | src/QPerl6/Actions.pm:
Fix meta-op code gen a bit.
kudo/toqast: 1a88ffa | jnthn++ | src/QPerl6/Actions.pm:
Translate various control structures over to QAST.
15:21 contingencyplan joined 15:59 dukeleto joined 16:04 dmalcolm joined 16:08 zby_home joined 16:13 jashwanth joined
pmichaud last week (or so) I ran into a problem where Rakudo would bump up against Parrot's recursion limit; I resolved that problem by having Rakudo bump up the recursion limit when it starts. 16:17
I have a related question now.
(and looking for ideas / advice)
when Rakudo or any Parrot program hits the recursion limit, Parrot throws an exception, as it's designed to do
the exception can then be caught by a handler
however, the handler is somewhat stuck, because it cannot turn around and make any subroutine or method calls, because those would also exceed the recursion limit 16:18
so.... what to do?
whiteknight pmichaud: interesting quandry 16:41
Honestly, I wonder if there's benefit in having a recursion limit at all, by default
or, if we hit that limit, add on a small emergency "bump" to enable error handling 16:42
pmichaud sorear++ mentions on #perl6 the possibility of a soft limit
moritz whiteknight: it's useful for detecting bugs 16:45
whiteknight moritz: yes, but if it's a source of new bugs, that defeats the purpose
or, if we had a limit we could toggle on and off 16:46
or a soft limit seems good too
(depending on desired semantics)
pmichaud this would be kind of off-the-wall, but one solution would be to have the 'recursion limit exceeded' exception also disable the limit (or set it very high). 16:47
the handler could then set it back down again after it's been handled
there might be some issues there that handlers not expecting the exception might not handle it properly, though.
(since the default is for a handler to 'catch everything' iirc)
moritz another crazy idea 16:48
pmichaud afk, kid shuttle 16:49
moritz only trigger the exception on an exact match o recursion depth, rather than >=
that way it's the user's choice whether ot unwind the stack after such an exception
pmichaud that could work also, except if somehow your handler leave things in a state where you're already nested above the limit
moritz well, then you won't have that protection in future 16:50
maybe something like (depth > 0 && depth % limit == 0) 16:51
parallel make is broken for me on parrot master 16:56
oh wait 16:58
stale Makefile
whiteknight if we had an exact == on recursion limit, we would get the exception again when the handler returned 16:59
we probably want a leading edge trigger, so it only throws when we hit it the first time while incrementing
moritz thinks he detects an electrical engineer speaking :-) 17:00
whiteknight ....no.... 17:02
dalek kudo/nom: 3bc91c1 | pmichaud++ | src/Perl6/Grammar.pm:
Recognize obsolete forms of rand() and rand(N), per STD.pm. Fixes RT #113968.
 17:37
17:52 birdwindupbird joined 17:56 dukeleto joined 18:48 kjs joined 18:49 lucian joined 18:55 brrt joined 19:20 rich joined 19:25 arnsholt joined 19:36 brrt joined 19:47 chromatic joined
chromatic pmichaud, ping 19:47
pmichaud chromatic: I have to depart for about 5 mins; will pong on return
chromatic I'll be here. 19:48
whiteknight chromatic! 19:50
chromatic Turns out the only time I have time to work on Parrot these days is when suffering migraines. Who knew? 19:51
19:54 alvis_ joined
chromatic Did you see the enhanced sweep 0 branch? 19:54
whiteknight I saw it go by in the feed, I haven't looked at the changes yet 19:56
19:56 brrt joined
chromatic Essentially all it does is recycle all PMCs in the youngest generation if they're marked with the early_gc flag. 19:56
That means you can write:
needs_destroy $Px 19:57
sweep 0
pmichaud pong
chromatic and reclaim $Px without having to do a full mark and sweep.
pmichaud, how much work is it to change one of your code generators (something in LTM might be a big benefit) with the manual escape analysis I mentioned in your GC ticket?
pmichaud chromatic: well, your ticket made it sound like a lot of work. 19:58
whiteknight chromatic: it looks .... dangerous
chromatic The hard part is figuring out the expected lifespan of a PMC.
whiteknight chromatic if I have scope Foo() with early pmcs, and then call into Bar() with early PMCs, it seems like a sweep 0 in Bar will kill referenced PMCs in Foo 19:59
chromatic Yeah.
pmichaud what could be helpful for debugging purposes is if I can mark a PMC somehow as "I expect this to be destroyed", and then get an exception (along with the PMC referencing it) if it ends up being marked. 20:00
then I could do "destroy_ready $Px" and "null $Px" and "sweep 1" and find out what is still referencing it.
chromatic That's maybe four lines of code. 20:01
pmichaud but really, my ticket was less about finding dead PMCs and more about avoiding the panic.
I don't see how "needs_destroy" would help my calling-many-subs example, for instance.
chromatic I had the mental energy to figure out alternate memory usage problems but not to avoid the panic.
Let me read that example again. 20:02
whiteknight the panic we can definitely avoid 20:03
it's just a small matter of code
chromatic Many subs we could fix if we marked recyclable continuations as early GCs.
We know when we're not storing those continuations.
pmichaud the problem is that *any* pmc allocation in that situation causes a panic.
where the situation is that parrot believes it should be allowed to allocate more memory (based on total memory size of the system), and the system ultimately says "No", even though Parrot is already using 200x more memory than it needs. 20:04
chromatic Where "needs" means "trading memory for time" of course.
pmichaud Yes, it may mean that. 20:05
chromatic On the extreme side of memory parsimony, we could do a GC run for every PMC allocation.
pmichaud but if $P0 = new 'Something' can cause a panic even when Parrot is using hundreds of times more memory than the working set actually requires... that seems off to me. 20:06
but I'm not wanting to debate in extremes here, or to argue about what a proper working set size should be.
chromatic The current tuning is "reach the end of a pool with a size of some tuned fraction of total memory, and then do a GC run".
whiteknight Old behavior, I think, was to do a sweep before allocating a new arena
kjs Lurking on this discussion, I'm reminded of a bit of code I wrote for M1. The code generator needs registers all the time to store numbers and results when evaluating expressions. It's really easy to run out of 240 registers very quickly. Instead, M1's code generator "freezes" registers when they are assigned to real symbols (declared as variables), whereas others aren't frozen. Then, whenever a register is seemingly no longer used, it's "free'd()"; t
"free_reg()" function checks whether the reg wasn't frozen (a symbol!); if it is, do nothing, if it isn't unuse the register. Same could apply: PMCs are needed for temp results, but at some point not longer needed.
whiteknight the newer behavior, of eating up memory until a threshold definitely trades performance for memory usage
maybe we need some kind of a flag to switch between the two behaviors 20:07
chromatic A flag adds complexity.
pmichaud what I do know is that on systems with limited swap or memory availability, having parrot panic when it really has other options available to it seems suboptimal.
chromatic Yeah, the panic is a problem in any system.
whiteknight a flag doesn't add nearly so much complexity
chromatic There ought to be a way to say "Try to get more memory, but if not, try to run the GC, and if that finally doesn't work, admit you're in trouble."
pmichaud that was my suggestion (here on #parrot), yes. 20:08
whiteknight yes, that is doable also (but doesn't add any less complexity than checking a flag)'
I'm cool with with whatever path people want to take
chromatic Ultimately the real improvement comes from what kjs said.
pmichaud I don't have a specific solution in mind. I know that we're getting more people trying to build rakudo and parrot stuff on small-memory systems (e.g., arduino, raspberry pi), and although they'll accept panics, it's hard to explain the reason why. 20:09
chromatic Some things are (in C or Perl 5 terms) stack/scope allocated. Some things are heap allocated. Telling the difference is important.
whiteknight so do we need a "destroy_this $Px" opcode, that a code generator can jam in where necessary? 20:10
pmichaud part of my subcall example was to point out that any pmc allocation could trigger the panic, not just those that might result in large data structures.
I see "destroy_this" as being more debugging than production.
whiteknight explicitly saying what to free and when sure beats any heuristic with false positives or negatives 20:11
pmichaud yes, but it can make for debugging hell, too.
chromatic Part of the motivation for needs_destroy is finalization, a la P5.
pmichaud yes, I understand that part of needs_destroy. For the stuff we're working with, getting code gen to recognize it seems problematic. 20:12
whiteknight okay, let's look at the root problem first. GC shouldn't panic on failed memory allocation if it is possible to perform a sweep and reclaim things that way 20:13
any other solutions are just window dressing
pmichaud effectively, part of the problem we have now is finding out the size of our data structures, or knowing where to prune. 20:14
whiteknight If an arena allocation fails we attempt mark/sweep. if mark/sweep fail, then we panic
pmichaud where "we" == "rakudo/nqp"
something is eating up lots of memory, and although we believe we're releasing the references to the parse tree, something is still keeping it alive.
whiteknight pmichaud: so a tool that traced out the object graph and wrote it to file would be very useful 20:15
pmichaud yes, that tool would be very useful.
whiteknight okay. That will be item #2 on my list
first is to fix this panic
pmichaud also, fwiw, the panic is not super-high priority to rakudo/nqp
I just encountered it while trying to come up with a response to a question on p6c 20:16
however, fixing the panic would enable us to better figure out what is happening on low-memory systems
whiteknight we haven't been getting a lot of feedback about priority TO-DO items lately, so anything we hear about has to be high on our list
chromatic Would that four-line patch I mentioned (throw an exception when marking a PMC flagged for early GC) be more useful?
whiteknight chromatic: that sounds like it, yes
pmichaud tracing the object graph would be most useful at the moment, I think. four-line-patch to signal when a flagged PMC is marked would be useful, too. 20:17
chromatic It's a lot simpler to write that small patch than to serialize the object graph.
pmichaud agreed.
chromatic That at least gives you the possibility to do some debugging, however manual. 20:18
pmichaud correct.
chromatic Alright, give me a few minutes.
pmichaud thanks.
chromatic whiteknight, I still think there's stuff to salvage from the early_gc branch.
whiteknight chromatic: yes 20:19
chromatic P5 has the notion of temp storage, which is essentially a stack.
If needs_destroy were associated somehow with contexts and sweep 1 respected that, we could avoid the problem you mentioned.
whiteknight chromatic: ah, that's a great idea. The GC could contain a list of context/pmc pairs in a stack-like list 20:20
wait, maybe not
chromatic Modulo the complexities of cycles in the continuation graph.
whiteknight Best we could do is free the early PMCs when the context was freed
because with continuations floating around, a context is not dead until it's really really dead 20:21
chromatic That might be even easier than repurposing sweep 1.
whiteknight last I heard, bacek was talking about deprecating the sweep and needs_destroy opcodes entirely
so our plans for those are not too aggressive. Any improvements or semantic modifications may be welcomed 20:22
chromatic There still has to be something at the PIR level to say "Finalize this as soon as possible".
whiteknight yes. Figuring out what that should be is very hard
I've spent much time on the topic. Lots of thinking, few results
chromatic pmichaud, do you prefer a Parrot exception or an assert? 20:25
pmichaud Parrot exception, likely. I think I need to be able to catch it so I can introspect whatever-it-is that is holding the reference. 20:26
chromatic Can do.
pmichaud although I suppose I could it in gdb
chromatic If PIR/NQP/whatever's more fun, a Parrot exception it is.
pmichaud but introspection at the PIR level would be nicer (i.e., I could Data::Dumper the thing, or at least query its class type) 20:27
chromatic My thinking too.
whiteknight With a quick sed script, we could change Parrot_gc_mark_PMC_alive to take two parameters, the parent and the child 20:29
After that, serializing the object graph would be trivial
probably would want to pass in a special flag to the build
pmichaud downstream users would need to switch it also, yes? 20:30
or are we using an improper API to mark there, then?
whiteknight I don't know. Need to think about this much more
this is a very clear case of more features == worse performance 20:31
chromatic I'm running the test suite now. 20:34
$ ./parrot pmc_early_gc.pir
GC_GMS marking a PMC flagged for early collection!
current instr.: 'go_crazy' pc 24 (pmc_early_gc.pir:10)
called from Sub 'main' pc 16 (pmc_early_gc.pir:5)
Hm, I need to put the offending PMC in the payload somehow. 20:35
pmichaud, I just pushed a branch with those changes for you. 20:56
dalek rrot/chromatic/early_pmc_mark_throws_exception: 051857b | chromatic++ | src/pmc/exception.pmc:
Added set_pmc_keyed_str() VTABLE to Exception PMC.
 20:57
rrot/chromatic/early_pmc_mark_throws_exception: 8896a87 | chromatic++ | / (3 files):
Added GC exception when marking early_GC PMC.

This may cause weird errors, depending on your use of the needs_destroy opcode. Because nothing currently sweeps up early_GC PMCs, they may exist somewhere in your object graph.
That does allow an interesting debugging possibility, however: mark such PMCs and catch the exceptions to see if you have references to them from places you don't expect.
Note that a couple of the t/op/gc.t tests fail. Those failures make sense to me, but the tests themselves don't. A little more sanity there might make the intentions more obvious.
21:14 brrt joined
brrt i basically need to get the Parrot_Interp from a Parrot_PMC 21:14
and it should work and not be ugly
and, mind you 21:15
not call any Parrot_api_* calls
pmichaud chromatic: btw, I don't know if I mentioned it at yapc::na, but we now have a start on a p6 "perldoc"-like tool: github.com/perl6/doc and doc.perl6.org/ . It's still in its infancy, of course, but it's started. 21:17
I'll look at the gc branch in the next day or so.
21:23 brrt left
dalek sella/gh-pages: 4ed6c17 | Whiteknight++ | / (2 files):
[Xml] Add stub doc file for Xml
 21:27
sella/gh-pages: c868c44 | Whiteknight++ | libraries/xml.md:
[Xml] fill in a few functions
sella/gh-pages: 92d4588 | Whiteknight++ | libraries/xml.md:
[Xml] Flesh out most of the documentation.
sella: b9e79c8 | Whiteknight++ | src/string/String.winxed:
[String] small doc addition
sella: d1d391f | Whiteknight++ | s (2 files):
[Utilities] Add a new rosella_doc utility, for creating markdown docs for Rosella.
sella: dd45683 | Whiteknight++ | src/query/Iterable.winxed:
[Query] Iterable.any() was broken. It's consuming an extra value, which disappears when we read the stream. Make this problem go away when we don't have a predicate, and put in a TODO note to properly fix it later
whiteknight brrt: I think the ParrotInterpreter PMC type has a get_pointer vtable which will automatically be called in NCI
brrt: what do you need it for?
dalek sella: bde7615 | Whiteknight++ | / (18 files):
[Xml] Xml is now stable.
 21:36
21:48 kid51 joined 22:44 preflex joined 23:41 tokuhirom joined 23:50 plobsing joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!