Parrot 0.8.1 "Tio Richie" Released! | www.parrot.org/
Set by moderator on 19 November 2008.
Infinoid pmichaud: thanks for taking that patch and running with it, sorry it wasn't very compatible to start with 00:00
jonathan pmichaud: I'm still concious.
pmichaud Infinoid: hey, it's great.
jonathan Let me try it on Win32 for you.
pmichaud jonathan: thanks.
I get an infinite loop in stm/running.t, though. 00:01
jonathan pmichaud: I thought you just said it passes all tests?
pmichaud ...except that one.
jonathan lol
pmichaud which I'm thinking we'll deprecate.
I'm not at all sure how to handle contexts and threads. 00:02
jonathan Where's it inf-looping, or is it hard to tell?
pmichaud test #4
jonathan Probably with difficulty, but that's a 1.5 problem..
pmichaud pmichaud@orange:~/parrot/lex4$ ./parrot --gc-debug t/stm/runtime_4.pir 00:03
Bug: Attempting to mark dead context 8122988
current instr.: 'parrot;STMQueue;main' pc 508 (t/stm/runtime_4.pir:228)
if I run it with -t1 I get different results.
if run from gdb and then I hit interrupt, it's in __kernel_vsyscall 00:04
jonathan And a step down from that?
bt
jonathan is curious 00:05
nopaste "Infinoid" at 96.238.213.50 pasted "Regardless of whether the refcounting is right or not, if you're gonna take a reference here, might as well get a debug statement for your trouble." (17 lines) at nopaste.snit.ch/14657
"pmichaud" at 72.181.176.220 pasted "bt of t/stm/running_4.pir" (48 lines) at nopaste.snit.ch/14658
Infinoid ooh, deadlock 00:06
purl deadlock is the big difficulty with open2 (or with doing this in any other way.)
pmichaud Infinoid: I didn't want to change the ordering until I understood the logic a bit better.
jonathan pmichaud: Looks like a deadlock.
Infinoid ordering? oh.
pmichaud but I will make it into a context_ref call.
00:06 Limbic_Region joined
Infinoid well, that's fine, I can just move it down 00:06
jonathan pmichaud: And it hits it in destruction.
It may be the more general destruction ordering issues we have.
pmichaud note that it *did* run into a dead context exception, though. 00:07
#9
that probably shouldn't have happened in the first place.
chromatic Yeah, it doesn't feel like destruction ordering.
jonathan Yes, it's exiting as a result of the exception.
Infinoid on my box, lex4 has test failures in t/compilers/pge/06-grammar.t, t/compilers/pge/pge_examples.t, t/compilers/tge/grammar.t, t/library/coroutine.t
pmichaud Infinoid: your box is...? 00:08
dalek r32968 | fperrad++ | trunk:
: [Lua]
: - update syntax, see RT #57428
diff: www.parrotvm.org/svn/parrot/revision?rev=32968
00:08 bacek_ joined
Infinoid gentoo linux x86-64 00:08
pmichaud ah, 64-bit.
Infinoid all dead context bugs
pmichaud could very well be.
yes.
Infinoid so that's one fewer failure than this morning... t/compilers/tge/basic.t seems to work now 00:09
pmichaud I think they're still somewhat "floating"
00:09 AndyA joined
chromatic Crazy talk: because all vtable entries written in C are written in pre-processed C, we could make them all take parameters from the current context. 00:10
Then the VTABLE_ macros just marshall the arguments into a new context.
Infinoid hmm... should I do a similar hack to Parrot_free_context to see what's calling that? 00:12
pmichaud ...what's calling what? 00:13
Infinoid what's freeing the context too many times
pmichaud I'm just setting a breakpoint on Parrot_free_context
Infinoid that's too easy. :)
jonathan pmichaud: Tests running on Win32, results in a bit.
pmichaud I get a lot of __Parrot_context_ref defined but not used warnings -- how to suppress? 00:14
00:14 Aisling joined
pmichaud maybe I'll just make it Parrot_context_ref_debug 00:15
and then the macro can call it.
that sounds good.
Coke fperrad++ # cleanup up my deprecation hackage.
Infinoid make it inline, or -Wno-unused-function 00:16
ok :)
pmichaud and we can breakpoint it.
Infinoid sounds great
dalek r32969 | pmichaud++ | lex4: 00:17
: Missed a ref_count increment (Infinoid++)
diff: www.parrotvm.org/svn/parrot/revision?rev=32969
nopaste "jonathan" at 85.216.157.73 pasted "win32 test results in lex4" (8 lines) at nopaste.snit.ch/14659 00:18
pmichaud okay, not too bad. 00:19
I wonder if t/pmc/complex.t segfaults.
jonathan It does give Bug: Attempting to mark dead context 654268
pmichaud okay, so it's throwing an exception. 00:20
(sigh)
jonathan Works with -G, as you may expect.
pmichaud so.... why would we get dead contexts in some environments and not in others? 00:21
that seems... odd
jonathan 64-bit vs 32-bit would seem understandable.
Or at least you could guess some dodgy pointer stuff.
But hmmm.
pmichaud maybe, but even there I don't know why.
jonathan pmichaud: Does that complex.t not fail for you? 00:22
pmichaud no, it does not.
it seems to me like we should be getting the same context allocations and deallocations, yes? 00:23
jonathan Yes
Hmm
pmichaud that's really dependent only on execution.
whether the dead context is detected would depend on when the gc sweeps are run.
jonathan --gc-debug does what? GC run after eahc op?
pmichaud I don't know exactly what --gc-debug does.
but I tried it without --gc-debug and it ran fine also.
jonathan Turn on GC (Garbage Collection) debugging. This imposes some stress on the GC 00:24
subsystem and can slow down execution considerably.
OK, this one is interesting.
Becuase it shows up with parrot t/pmc/complex.t
but goes away with --gc-debug
pmichaud chromatic has a magic runcore that does a gc after every op -- I forget the option, though. 00:25
chromatic --runcore=gc-debug
pmichaud that's different from --gc-debug?
Coke yes
chromatic Sadly.
pmichaud Understood.
purl Understood. are you on schedule?
Coke no, understood is <reply>Roger Roger.
pmichaud pmichaud@orange:~/parrot/lex4$ ./parrot --runcore=gc-debug t/pmc/complex.t 00:26
main: Unrecognized runcore 'gc-debug' specified.
jonathan main: Unrecognized runcore 'gc-debug' specified.
oh, dupe fail
pmichaud: it's gcdebug
no extra -
Coke I blame chromatic.
jonathan My wob it's slow.
Coke ... hell yes.
it's INSANELY slow.
pmichaud 150 tests....
180 tests... 00:27
jonathan It's this slow with virtually nothing to collect?!
oh arse
We run to completion with that too.
So something gets collected that would reference a context when it's OK to mark it...
pmichaud then I'm suspicious of whatever --gc-debug does.
or what you just said. 00:28
jonathan ...but lasts longer normally at which point it becomes un-OK to reference that context.
pmichaud oh, so perhaps my dead context detector is broken. 00:29
no, that can't be it.
"lasts longer normally" would mean that it still wouldn't mark its pointers as live, or follow them.
jonathan No, I don't think so.
Moment, will try and debug 00:30
pmichaud okay.
jonathan OH PLEASE NO 00:32
It runs to completion under the C debugger too.
pmichaud I suspect --gc-debug.
jonathan Yes, but that hid it.
Oh, you mean you think that'd flag would be enabled?
Shouldn't be.
pmichaud no, I just mean that whatever --gc-debug does is changing things somehow... what does --gc-debug do, internally? 00:33
jonathan No idea.
chromatic I think it mostly turns off garbage collection.
jonathan But it didn't fail with --runcore=gcdebug either
pmichaud turning off gc stresses the gc subsystem? 00:34
is --gc-debug a separate runcore, or ...?
chromatic Never recycling any memory stresses the system.
jonathan pmichaud: No, --runcore=gcdebug
Is the separate runcore. 00:35
pmichaud stresses the system, yes, but stress the gc subsystem?
jonathan Yes, because without recycling of stuff there's more to collect.
Infinoid is it likely that a context's lex_pad pointer would point to a Null PMC? 00:41
pmichaud yes.
if the sub doesn't declare any lexicals.
Infinoid I'm looking at a dead context crash in gdb, and trying to make sure the ctx is at least a valid ctx, and not some other random pmc
ok, thanks
this is strange. the context's ref_count is -1 (hence the dead context crash), yet when I put a conditional breakpoint on the ref_count decrement, I don't find the smoking gun. 00:43
telling gdb to "break src/gc/register.c:534 if ctxp->ref_count <= 0" should work. And it does trigger, if I omit the condition.
pmichaud there are two refcount decrements 00:44
be sure to hit the second.
Infinoid well, that would explain it. 00:45
I've done this just enough times to cringe whenever I see IMCC in the backtrace...
ok, so this hit the "force the reference count negative to indicate a dead context so mark_context (src/sub.c) can report it" sanity check. yet the context is apparently still around, somewhere 00:47
pmichaud yes. 00:48
dalek r32970 | pmichaud++ | lex4: 00:50
: Suppress __Parrot_context_ref unused warnings
: (cleans things up a fair bit overall, too).
diff: www.parrotvm.org/svn/parrot/revision?rev=32970
00:51 TiMBuS joined
Infinoid sets a bunch of read-watchpoints in hopes of finding out where else the context lives. 00:53
00:55 Alias joined
chromatic Instead of recycling contexts onto a free list, you could temporarily add code to malloc/free them, and catch the segfault. 00:56
pmichaud they're already malloced, so it's just a matter of freeing them I suppose. :-) 00:57
jonathan There's so memory corruption at play somewhere, I fear. 00:58
chromatic Yeah, free them instead of putting them on the free list.
Good for debugging.
jonathan When adding one line of C that just follows a pointer to try and get a segfault then makes this test run to completion...I get suspicious.
chromatic Hm, I would too. 00:59
01:01 davidfetter joined
Infinoid there's a Sub that still has a reference to this (now freed) context 01:01
pmichaud what pointer? 01:02
purl pointer is, like, NULL :-)
pmichaud purl, forget ointer
purl pmichaud, I didn't have anything matching ointer
pmichaud purl, forget pointer
purl pmichaud: I forgot pointer
Infinoid #3 0x00007f4da50d184d in Parrot_Sub_mark (interp=0x185d080, pmc=0x193e3d8)
at ./src/pmc/sub.pmc:443
443 mark_context(interp, sub->ctx);
pmichaud which test is this, the stm one or...?
Infinoid no, this is just trying to compile PGE after adding chromatic's free() (which is a great idea). 01:03
though I saw something very similar a little while ago, by breaking on the dead context exception and doing a backtrace there 01:04
(though I guess that doesn't mean much, since every sub has a context)
jonathan Go Microsoft. The latest version of Visual Studio won't even let me open the EXE for debugging.
Infinoid debugging is far too boring, and doesn't appeal to middle management 01:05
Alias :) 01:06
And have you ever tried to develop a PKI for debugging so that it can appear on the BIS status dashboard? It doesn't deserve to be considered by any modern CIO 01:07
Infinoid goes into TLA overdose 01:08
chromatic It's worse to realize you understand the whole thing, and you don't even write business software. 01:09
jonathan Debugging mode seems to do something different with memory allocations somehow. Which makes the problem vanish.
Which makes me suspect memory corruption all the more, but gives little aid with finding it.
in mark_context 01:11
This is maybe unrelated but obj = (PObj *)interp->current_cont; 01:12
pmichaud mark_context really only helps us detect that something went awry; it doesn't really pinpoint the problem.
jonathan That is, we mark something in the interp data structure every call to mark_context?!
Shouldn't we be doing that once per DOD?
chromatic I would think so.
jonathan Won't be this bug, but it looks so wrong.
pmichaud I think mark_context might've originally designed when contexts were always a stack. 01:16
but yes, it looks wrong. 01:17
Alias OK, me test failure 01:23
my
Inside the Perl::Dist build chain, on XP current service pack, I see t/pmc/complex failing 01:24
This is the 0.8.1 release
jonathan Alias: That's due to floating point errors, I guess?
Alias uuuum
pmichaud ...the release, not the trunk?
Alias The release, yes 01:25
0.8.1 release
jonathan OK, by making contexts just always malloc some memory rather than take from the pool, I can get the problem to trigger in the debugger.
Alias There's 2 subtests failing
I can't tell you which, no output
jonathan Alias: Aha. We're debugging a new problem in a branch at the moment, which gives a different, unrelated failure here.
pmichaud ...is it unrelated, though? 01:26
jonathan But I do know and have seen the failures.
pmichaud interesting that they both show up in complex. :-)
jonathan pmichaud: Yes, I've seen the failures that I think Alias mentions. They were -0.0 vs 0.0 related.
pmichaud okay.
jonathan That test has failed for me with those on Win32 for quite a while.
Alias STDERR dump says, "Have: 0.000000-0.909297i\\nWant: -0.000000-0.909297i"
jonathan Aha.
Yes.
pmichaud yes, that's it.
Alias And another similar pair
With the centre "-" replaced with "+" 01:27
So whatever those two are...
jonathan pmichaud: Ugh. The thing calling the mark_context I get this in is the mark routine in RetContinuation.
Alias jonathon: Anyways 01:28
Am I right to leave that with you guys and come back next release? :)
chromatic Not unless you also leave with us a Windows programmer who knows how to fix transcendental math on Windows. 01:29
jonathan Alias: The problem you're seeing is actually still a problem.
Alias Oh, and one you can't fix?
jonathan I'll bet once this issue is fixed, that one is still there.
Alias (trivially)
jonathan Alias: I don't, myself, know how to fix that floating point one.
chromatic Not trivially, no.
jonathan If you do know about that area, help welcome! :-) 01:30
Alias I don't know C, alas
jonathan You lucky thing!
Alias I just know how to take everyone else's knowledge and automate them into build systems
(Because as I see it, the problem with most build systems is they are created by sysadmins) :) 01:31
jonathan aha 01:32
/*
* XXX when reusing SUPER.destroy() RetContinuations
* have to set ref_count initially to 1
*/
Alias Later today I might try to add force support to my parrot build
jonathan In RetContinuation PMC
Alias And then I'll just continue to the install step
And see if I can at least produce a real .exe installer
jonathan oh, but we don't call that :-|
Alias Will get back to you once I have a package
jonathan Alias++ # packaging
Alias ARe you guys happy with just something that installs to C:\\parrot
chromatic Alias, that suits me for now. 01:34
Alias And now I go back to $work fighting with rpmbuild
chromatic: OK
I'm not sure wtf to do with the environment stuff, but we can deal with next next iteration
jonathan pmichaud: Is Parrot_free_context a macro? 01:36
pmichaud yes 01:37
no.
(sorry, was thinking Parrot_context_ref)
Parrot_free_context is in src/gc/register.c
Alias BTW 01:38
If you wanted to fix that math issue
What, PRECISELY, would you need
I can mention it to my Microsoft OSL contact, and see if they can help
Assuming that the problem is really deep Windows-specific stuff 01:39
jonathan ah, we're hitting
/* force the reference count negative to indicate a dead context
* so mark_context (src/sub.c) can report it */
ctxp->ref_count--;
chromatic We would need a Windows-specific way to detect negative floating point zero and always force output of that value to appear with a leading -. 01:40
jonathan pmichaud: Aha
Parrot_free_context(PARROT_INTERP, ARGMOD(Parrot_Context *ctxp), int deref)
We do
if (ctxp->ref_count <= 0) {
Alias erk 01:41
jonathan oh, ah
Alias chromatic: Can you fire me off an email with the details and anything you think might be useful for someone on the inside?
jonathan false alarm
pmichaud jonathan: right.
:-)
Alias chromatic: And I'll forward it through
jonathan Gah, how is this one happening... 01:42
chromatic Will do.
Alias Thanks
No promises, but it's worth just lobbing it over the wall and seeing
chromatic That might get us a better fix than hoping someone will show up with a patch.
Alias Did you ask the mingw people? 01:43
chromatic Don't know any.
Alias I guess I don't know enough about how numbers relate to OS
Cause it sounds like a compiler thing to me, not so much OS 01:44
chromatic I think it's a libc issue.
01:46 davidfetter joined
Alias So that makes it more of a MinGW thing? 01:46
BTW, after make test, I should just make install it? 01:47
chromatic Whatever the standard math functions in the standard C library do, that's what we have to work with.
Alias hrm
ok, I might try pushing to a newer version of MinGW than I use for Perl 5 01:48
chromatic We're talking about the representation of newer values.
I mean "numeric" values.
Alias ok
chromatic I'll try to make it clear in the mail.
Alias Thanks
I'm good at chasing people down for stuff, so I'll see what I can do
And in the mean time, will upgrade my C toolchain more aggressively for parrot builds
chromatic_suppertim Alias, if you're bundling 0.8.1, you might add in r32929. 01:51
Alias My setup isn't really able to support patches
Because it's designed for building production-grade releases, so it mostly insists on sticking to only what comes in the tarball 01:52
Which is the result of one of the original design choices for Strawberry
I can just pick up the next one in a month
chromatic_suppertim Fair enough. If you did ever pick up a patch though, that's the top on my list. 01:53
Alias I only allocate budget to hack on the Perl::Dist code itself once every 3 months 01:54
time budget...
If something comes up in January, I'll see :) 01:55
But then I expect a new monthly release by then
confound supertim 02:15
chromatic_suppertim suppertime
Whiteknight suppertime!!
(you have to say it with more gusto
davidfetter bon appetit, chromatic_suppertim 02:16
jonathan pmichaud: Mighta found something. 02:20
Something that isn't my bed.
Yes. It fixes t/pmc/complex.t here 02:21
pmichaud: Testing, then will ci 02:22
Then will sleep.
jonathan wonders if he might as well just stay living on PDT...
2 hours for a one damm line patch. 02:32
dalek r32971 | jonathan++ | lex4:
: [core] We weren't ref-counting contexts created in PMCs that had methods using the Parrot Calling Conventions the same way as with normal Parrot sub calls. Bring the two inline. Clears up a failure on Windows.
diff: www.parrotvm.org/svn/parrot/revision?rev=32971
Alias jonathon: That is the way of things... 02:33
The reason it took 2 hours to find was BECAUSE it was so small
pmichaud looks at jonathan++'s patch 02:35
jonathan hopes it's sane 02:36
pmichaud I have no way of truly knowing. But yes, you're correct that it doesn't match Parrot_PCCINVOKE
or Parrot_pcc_invoke_sub_from_sig_object 02:37
jonathan Right.
pmichaud so if those work, then that one must need to be there as well.
I'll test on my box
w/realclean
jonathan I followed what they did side by side, once I got to the point of being pretty sure it was to do with both ret continuations and PCC methods.
And realized this was the messing step.
One of the STM failures went away. 02:38
pmichaud oh, yay!
jonathan The other one that fails I think normally fails on Win32.
pmichaud I'll be able to tell in just a bit.
jonathan Cool 02:39
jonathan crosses his fingers
pmichaud if this is it, in some sense it's amazing we hadn't noticed problems before now. 02:41
(running 'make test' now.)
sometimes, though, adding a refcount actually hides a problem instead of solving it (by turning it into a context leak) 02:42
jonathan Well, it might always be one of a number of issues.
pmichaud but hopefully this will allow me to get rid of my known leak.
(which, when I remove it, causes PGE to stop building.)
so far all is well. 02:43
jonathan figures he's stayed up late enough that waiting for the make test results won't hurt any more 02:45
pmichaud all tests passed. 02:46
_including_ the stm one that was hanging before.
jonathan OK
That sounds like good news.
pmichaud indeed, it does. Now let's see what happens when I get rid of my context leak. 02:47
FAIL. 02:48
jonathan Damm. 02:49
So are we better off without the patch, or better with it, or...?
pmichaud with it.
jonathan OK
So it's fixed one problem but we still have another?
Infinoid lex4 passes tests on x86-64
pmichaud yes, there's a memory leak somewhere.
Infinoid jonathan++ pmichaud++
pmichaud Infinoid: excellent. 02:50
jonathan pmichaud: A merge-blocking one?
Or a minor one?
pmichaud well, I'm not a fan of leaking contexts.
jonathan Sure.
pmichaud and we haven't tried rakudo in the lex4 branch yet.
jonathan Ah.
pmichaud (trying that now.)
Infinoid oh, we still leak
valgrind time.
(that's like hammer time, but a bigger hammer.) 02:51
pmichaud yes, the noop.pir test leaks again.
that one is...tricky
jonathan It didn't leak before the patch?
Or again, related to the other problem?
pmichaud before your patch? it leaked _also_
jonathan OK.
pmichaud so I think the leak is unrelated to your patch.
jonathan Yes.
Seems so. 02:52
pmichaud I think it's related to the from_ctx pointer.
jonathan What's noop.pir?
pmichaud .sub main
noop
.end
jonathan Ah.
pmichaud leaks 112 bytes on my system (one context, I think)
tracing shows two references made, only one pop
jonathan OK, and if you do many calls of such a sub, do we leak per call?
pmichaud er, only one free
haven't tried that yet.
jonathan OK, that'd answer a lot.
pmichaud rakudo passes "make test" 02:53
(trying "make spectest" now)
chromatic_suppertim Interested in Parrot: matt.might.net/
jonathan Yes, same here.
(make test)
Infinoid oh. these things are never freed, they're just put in a reuse pool, right?
pmichaud correct, they're in a re-use pool.
Infinoid explicitly frees them again, to make valgrind work better 02:54
pmichaud I've taken to calling it "the recycle pool".
jonathan OK folks, I really should sleep. 02:55
pmichaud yes, you should. It's even getting late-ish here.
Infinoid t/compilers/pge/06-grammar.t is a pretty good test for this
jonathan Will hack more stuff tomorrow.
Infinoid goodnight jonathan 02:56
pmichaud Thanks for taking the time to track down that one bug.
jonathan night
Infinoid ==27842== 1,062,696 (519,792 direct, 542,904 indirect) bytes in 2,012 blocks are definitely lost in loss record 1,262 of 1,262
==27842== at 0x4C21FAB: calloc (vg_replace_malloc.c:279)
that's gotta be the leak.
jonathan np, I want to see this branch merged as soon as anyone
Infinoid ==27842== by 0x510F689: mem_sys_allocate_zeroed (memory.c:102)
==27842== by 0x510FFFA: Parrot_alloc_context (register.c:447)
jonathan ouch
jonathan sleeps
Infinoid valgrind has all kinds of leaks, categorized by the functions on the stack 02:57
02:57 jimmy joined
Infinoid the biggest source of unfreed contexts comes from Parrot_callmethodcc_p_sc, via Parrot_Sub_invoke 02:57
pmichaud ...callmethodcc ?
you mean Parrot_sub_invoke is calling callmethodcc ? 02:58
nopaste "Infinoid" at 75.28.77.95 pasted "Biggest source of unfreed contexts, according to valgrind" (16 lines) at nopaste.snit.ch/14661
pmichaud oh, callmethodcc is invoking Parrot_Sub_invoke 02:59
Infinoid of course, valgrind doesn't tell us why they aren't being cleaned up
pmichaud I still think it's something to do with from_ctx (in Parrot_Sub_invoke) 03:00
Infinoid is that the same from_ctx that I was staring at earlier, in the continuation pmcs?
pmichaud no, this one is in src/pmc/sub.pmc 03:04
currently I give it a context_ref..... but I can't see that it's ever freed.
if you run the noop test, you'll see that only two references are made, and that's the one that isn't freed.
(but if I take that ref out, then other things go kaboom.) 03:06
Infinoid ok, you're right, the vast majority of leak traces go through Parrot_Sub_invoke 03:08
nopaste "pmichaud" at 72.181.176.220 pasted "easy to see leaking script" (14 lines) at nopaste.snit.ch/14662
pmichaud that script leaks 100 contexts. :-)
or maybe 102.
Infinoid the other big leaker seems to be count_signature_elements, but the numbers for that one aren't quite as big
pmichaud here's the line I'm talking about: src/pmc/sub.pmc:264 03:11
I can take that out and then PGE doesn't build. 03:12
(but we don't get the leak.)
Infinoid hmm... if Parrot_free_context is never being called, why would the refcount matter? 03:14
pmichaud it's called once, but there are two references.
and it should be called at the end of the program when everything gets cleaned.
(apparently that's not happening, which is why we end up with leaks)
Infinoid (for the record, I added one at the end of the invoke method, and it broke PGE.) 03:15
pmichaud (two references means it should be called at least twice)
Infinoid yeah... I only saw the one
and I thought ref_count was initialized to 0
I'm probably wrong about that.
pmichaud it is initialized to zero to begin with.
anyway, my updated noop.pir script that runs 100 invokes definitely shows we have a leak somewhere. 03:16
Infinoid anyway, PGE is segfaulting because of a dangling pointer to garbage, after an 18-context marking chain in a DOD run 03:17
yes, definitely
pmichaud a-ha 03:18
src/pmc/retcontinuation.pmc:94 03:19
, 0) should probably be , 1)
although that also causes the PGE build failure. 03:20
Infinoid hum.
pmichaud for background: in trunk the from_ctx pointer does not appear to increase ref_count
Infinoid 12:52 <@particle> and retcontinuations are single-use, so never incremented/decremented
12:52 <@particle> right?
12:52 <@Infinoid> is new_continuation aware of that, so it never takes the reference?
12:52 <@chromatic> Yes and I don't know, respectively. 03:21
that exchange led me to believe the refcounting was right
pmichaud however, in trunk that Parrot_free_context *does* decrement the refcount pointer.
03:21 Andy joined
Infinoid oh. 03:21
03:22 Psyche^ joined
Infinoid works on hunting down the details of the segfault 03:23
pmichaud so, at first I was going with "don't refcount the from_ctx", but got the segfault.
then I tried "okay, de-refcount the from_ctx in retcontinuation", and that segfaults. 03:24
I lean towards not refcounting from_ctx, but....
Infinoid yeah... I'm not sure the segfault is directly caused by this yet
pmichaud I need a break for a bit -- bbi30 03:25
03:35 Theory joined
pmichaud well, looks like it's going to be closer to an hour :-( 03:48
Coke notes that jonathan, pmichaud, and Infinoid all have the same # of letters in their nicks, so they all line up. 04:24
Infinoid It's Super Fun. 04:32
pmichaud particle too. :-) 04:37
okay, it's time for me to learn "Everything You Ever Wanted to Know About Parrot Continuations and Probably Shouldn't Have Asked". 04:38
a-ha! 04:46
the from_ctx pointers of continuations are refcounted. the from_ctx pointers of ret_continuations are not.
chromatic Do ReturnContinuations return to from_ctx?
pmichaud checking
yes, it appears so. 04:47
oh, maybe not.
I need to check further.
but in invalidate_retc_context, we convert all of the retcontinuations into continuations *and* update the refcounts for from_ctx
Coke is scared by the new star trek trailer. 04:48
chromatic Spock has The Hunger. 04:49
pmichaud invoking a RetContinuation switches back to the context given by to_ctx (Parrot_continuation_rewind_environment) 04:50
I suspect it doesn't need a from_ctx after that, which is why it's not refcounted.
also, Continuation's destroy frees its from_ctx pointer, while RetContinuation's does not (unless there's an implied SUPER that I'm not aware of) 04:51
(I'm looking in trunk for all of this information.) 04:52
chromatic There should be no implied SUPER; it has to be explicit. 04:54
pmichaud okay, good. This confirms what I'm thinking then.
so, why do Continuation's need to refcount their from_ctx, I wonder? 04:55
that seems backward. I would expect that we would need to refcount the to_ctx 04:57
oh, it appears to do with argument passing. 04:59
Coke chromatic: did a callgrind run on a simple tclsh program. 05:06
backs up my previous -p attempt on a different file: lots of time in gc. :| 05:07
nopaste "coke" at 193.200.132.135 pasted "cg.out" (779 lines) at nopaste.snit.ch/14664
"coke" at 193.200.132.135 pasted "foo.tcl" (5 lines) at nopaste.snit.ch/14665 05:08
Coke looks like hash is another big use of time. 05:12
has anyone investigated eliminating our hand rolled hash library and using an OS one?
chromatic No. That might be nice. 05:13
Coke I figure we are unlikely to do better at a standard hash than someone who's been concentrating on that.
chromatic Agreed. 05:16
05:25 Theory joined 05:56 Theory joined
chromatic has a naive MMD caching patch. It's ugly, but it does appear to speed things up. 06:09
I saw a paper somewhere about type lattices doing even better, but this is a start. 06:12
GeJ Ok, well... time to get my feet wet. I'm thinking about writing a series of blog posts about a "Extremely Beginner's Guide to Parrot". The basic idea would be to start at the PIR level as an introductionary language and climb the layers up to NQP. 06:22
chromatic That would be great! 06:23
GeJ Does the general audience here think that such a series would make sense? is this a duplication of an already existing effort?
Actually, it came to me when I realized that besides running the tests and reporting failures I have done zilch for the project. 06:25
So I see this series as a way for me to learn Parrot and hopefully do something useful in the future. 06:26
Christian Aperghis-Tramoni has made a series of articles for a French magazine, but he did that at the PASM level IIRC. 06:31
It was last year I believe. I was thinking about doing something similar but starting with PIR and including all the changes that happened since then. 06:32
06:52 Theory joined 07:08 uniejo joined
lathos Starting with p 07:13
PIR would be better, yes.
chromatic Hm, that cache doubled the speed. Nice. 07:44
moritz chromatic: there's another speed task coming ;) Rakudo's 'make spectest' seems awefully slow in the lex4 branch 07:45
after 7 minutes it just reachec S05 07:49
chromatic This MMD cache will speed that up too. 07:51
07:51 apeiron joined, elmex joined
moritz I hope so. 07:51
chromatic Looks like the naive cache is about 33% faster. 08:02
Maybe 35%.
moritz that won't account for the factor 3 (or larger) slowdown in the lex4 branch 08:04
chromatic No, but it's a very naive cache and I'm sure we can get vtable performance much higher with some clever thinking. 08:07
I might be able to squeeze another 5-7% out of it by freeing one of my temporaries.
moritz it still feels like there's something smelly going on in lex4 + perl6 08:08
chromatic Wouldn't surprise me. 08:10
08:21 tomyan joined 08:22 tomyan1 joined 08:27 apeiron joined 08:57 alvar joined 09:35 barney joined 09:37 iblechbot joined, Hadi joined 09:43 samlh joined 09:48 ff-wonko joined 10:01 mberends joined 10:03 ff-wonko joined 10:20 Hadi left 10:23 Theory joined 10:36 ff-wonko joined
cotto barney, I think you're right about a proper Pipp grammar having to use variables to keep track of how tags need to match. 10:56
I'm working on a test language to make sure I know how to make it work.
I was hoping to avoid dropping down to PIR, but I can't really expect elegance in a PHP parser, can I? ;) 10:57
11:08 Theory joined
cotto PCT is surprisingly easy. pmichaud++ 11:12
barney cotto: It would be nice to avoid keeping track of tags, but I fear so 11:16
moritz what's different in the case of tags then, say, parsing quotes as "..." and '...'? 11:17
barney when you have a consise example, you could add it to t/compilers/pct/complete_workflow.t
moritz s/then/than/
barney moritz: Inlined HTML is treated as a single echo statement. This effectively changes the tag style 11:19
moritz ouch
cotto s/HTML/text/ 11:20
(which means that we at least don't have to care about html parsing)
barney, why would an example in complete_workflow.t be useful? 11:21
barney We might have to care about HTML parsing, for things like session management
cotto I don't see the connection.
barney Just for reference. Or as an early indicator when a PCT feature, that Pipp needs, is broken 11:22
moritz (for the record) a few hours earlier I reported a huge slowdown in the lex4 branch. Now I can't reproduce it anymore, I guess there was something wrong with my laptop instead
barney I started complete_workflow.t because I thought that I had run into a PCT bug 11:23
moritz barney: why does session management needs HTML parsing? Usually it's done through cookies, which are transported in the HTTP header
cotto barney, I don't think a working example will need anything other than inline PIR, which I wouldn't expect to be particularly buggy.
barney moritz: The PHP session extension checks the availability of cookies. When cookies are turned off, than a session id is inserted in all internal links 11:25
barney likes simple examples
11:25 Theory joined
cotto barney, where's that documented? 11:26
I hadn't heard of that before.
barney de.php.net/manual/cs/ref.session.php 11:30
Probably a filter hook for extensions will suffice
11:34 Ademan joined 11:35 kj joined 11:52 Theory joined 11:54 Ademan joined
dalek r32972 | bernhard++ | trunk: 11:56
: RT#60682: [PATCH] rewrite of subclass.t to PIR
: Courtesy of Bruce Stockwell.
diff: www.parrotvm.org/svn/parrot/revision?rev=32972
12:02 gaz joined 12:14 ff-wonko joined
dalek r32973 | bernhard++ | trunk: 12:15
: [Pipp] Rewrite rule 'singlelinecomment'
diff: www.parrotvm.org/svn/parrot/revision?rev=32973
12:27 jimmy joined
dalek r32974 | bernhard++ | trunk: 12:31
: [Pipp] add some comments in grammar.pg
diff: www.parrotvm.org/svn/parrot/revision?rev=32974
12:31 davidfetter joined
cotto barney++ #"in most cases broken HTML' 12:35
ok. When I can't match quotes it's time for bed.
night
12:36 ff-wonko joined 12:46 rkw joined 12:52 Theory joined 12:56 ff-wonko joined 13:05 cout joined 13:07 Lorn joined
dalek r32975 | bernhard++ | trunk: 13:15
: [t] Simplify t/examples/namespace.t
diff: www.parrotvm.org/svn/parrot/revision?rev=32975
r32976 | pmichaud++ | lex4: 13:16
: Nothing else refcounts the from_ctx for retcontinuations, so we
: probably shouldn't either. This eliminates at least one memory
: leak but means that PGE doesn't compile, so we need to track that
: down.
diff: www.parrotvm.org/svn/parrot/revision?rev=32976
pmichaud (slowness of perl6 in lex4 branch) -- I suspect that's because of memory stress due to context leaks. 13:18
every subroutine call was eating up an extra 100 bytes or so. 13:19
moritz I've tested it again a few hours ago, and had only 10% slowdown or something 13:20
(which could be the MMD speedup in trunk by chromatic++, actually)
13:20 ff-wonko joined
pmichaud did chromatic commit his speedup? I didn't see it above 13:26
moritz I think not the MMD cache, but a different, smaller one 13:27
r32963 "This gives a modest 5% speedup on very MMD heavy code" 13:28
pmichaud ah. That is probably not in the branch yet. 13:29
jonathan Ah, chromatic did an MMD cache? 13:30
jonathan was going to hack on that today...
pmichaud whatever r32963 says
moritz jonathan: I don't know, but he didn't commit a cache yet 13:31
pmichaud lex4 branch has another segfault/dead context issue
jonathan No, that's not a cache, that's just temporary PMCs being used.
The same one being hunted last night? Or a new one?
pmichaud new one.
purl it has been said that new one is generated.
pmichaud it would be interesting to see what failures others get. PGE doesn't build on my system, but more usefully some of the tests in t/op fail now. 13:32
...including t/pmc/complex.t :-) :=) 13:33
jonathan pmichaud: With context related issues? 13:34
Are just floating point ones?
moritz t/op/debuginfo.t seems to hang
pmichaud undoubtedly context-related
yes, I get hangs if I leave the normal context recycling in place. I'll commit the 'free' patch. 13:35
dalek r32977 | pmichaud++ | lex4: 13:36
: Free contexts instead of recycling them in a pool. This makes it easier
: to catch some segfaults.
diff: www.parrotvm.org/svn/parrot/revision?rev=32977
nopaste "pmichaud" at 72.181.176.220 pasted "prove -r t/pmc in lex4 branch r32797" (12 lines) at nopaste.snit.ch/14668 13:39
"pmichaud" at 72.181.176.220 pasted "prove -r t/op in lex4 branch r32797" (7 lines) at nopaste.snit.ch/14669 13:41
jonathan We fail to build PGE on Win32 too.
pmichaud right
I let it build to the PGE failure, and then run prove directly 13:42
jonathan I need to do some non-Parrot stuff for a bit
pmichaud okay.
jonathan But will post results of the tests when they've run
Coke ~/always look on the bright side of life /~ 13:45
davidfetter i love the smell of monty python quotes in the morning 13:46
Coke has just found the monty python channel on youtube.
we should definitely link to this on our website: 13:47
www.youtube.com/watch?v=4vuW6tQ0218...re=related
parrot?
purl it has been said that parrot is our teacher, our mother, our secret lover or the reason Dan started or the reason Dan left or pretty onionish:)
Coke parrot is also www.youtube.com/watch?v=4vuW6tQ0218...re=related
purl okay, Coke.
jonathan pmichaud: Here, lexicals.t hangs 13:52
pmichaud did you try 32977 instead?
jonathan I svn up'd sicne the last commit, I think
pmichaud (It actually mem_sys_free's the contexts instead of recycling them in pool)
ah, object-meths.t #26 is nice and short. 13:54
...and it segfaults.
that looks traceable. 13:55
nopaste "jonathan" at 85.216.157.73 pasted "epic fail" (51 lines) at nopaste.snit.ch/14670 13:56
pmichaud well, the t/compilers/pge tests are pretty obviously going to fail. :-)
jonathan True 13:58
pmichaud oooh, object-meths.t fails after only two iterations. This is even better.
...and it fails without the tailcall. even better. 13:59
it always bugs me that turning on tracing (-t1) changes the results so substantially. 14:02
I wish I understood the rationale behind RetContinuations 14:07
oh. 14:11
14:13 mj41_ joined
Coke -t1 changes things? like -t4 used to? 14:14
pmichaud I had a program -- running without -t1 segfaulted after two iterations, running with -t1 segfaulted after 87. 14:15
Coke if was a GC related segfault that doesn't surprise me at all.
if it was not, then, yah, that's bad. 14:16
Coke kicks off a tcl spectest run to see if chromatic's fix helps.
s/fix/5% speedup in some cases/
pmichaud gc related, yes. 14:17
Coke well, the stuff in -t1 has to take up GC resources; unfortunate, but inevitable. 14:19
but if your program has a GC bug (goes away with -G) you have other debug options.
I've had pretty good luck using chromatic's posts... moment. 14:20
pmichaud tis okay, I'm past that one now.
Coke www.oreillynet.com/onlamp/blog/2007..._in_p.html
pmichaud I now know _what_ is causing the gc error, but I don't know what to do to fix it
Coke wozzit? 14:21
pmichaud retcontinuations don't increase the reference count on their context
as a result if anything else releases the context, it gets freed immediately (even though the retcontinuation still needs it)
jonathan pmichaud: We can make RetContinuations increment the count too, I guess, so long as they also dec it when they go away. 14:22
Coke now, without looking at the code, wouldn't the fix be for retcons to .... right.
no?
pmichaud I don't understand why they don't in the first place.
jonathan pmichaud: Probably a stupid attempt to be "optimal"
Coke bug?
purl i heard bug was www.cbttape.org/funny/bug3.jpg
Coke purl, you funny. 14:23
purl Coke: what?
pmichaud but yes, we can have retcons increment the count, and we have to remove the fact that the refcount increments when a retcon is promoted to a continuation
jonathan Yes
pmichaud (and I don't quite understand why we have retcons in the first place -- another attempt to be "optimal"?)
I'll try incrementing the refcount
this also explains why jonathan needed the fix he did last night
well, having retcons increment the count causes lots more failures. :-( 14:25
Coke if chromatic's fix gives me 5% improvement in speed, it'll shave -10 minutes- off the spec test run.
if someone with IMCC-fu makes a patch for #36283, I can fix the build/make test/languages. 14:32
kj Coke: that ticket can't be fixed easily 14:33
hello, btw :-)
mmm, maybe it can... 14:36
ah I think it mightn't be hard after all :-)
pmichaud theory: retcontinuations assume that nothing else will reference their from_ctx, and that if anything does reference the from_ctx then the retcontinuation gets converted to a continuation 14:38
bbiab 14:40
14:50 gryphon joined
Coke kj: i was able to get a patch that errored out whenever the = was used, but couldn't quite find the generated C code that kept track of what was inout. 14:51
kj: (might not be) excellent.
if you can do the delicate work there, I can do all the grunt work of editing PIR code.
kj Coke: working on something
Coke: should only IN operands be disallowed
so is OUT, and INOUT fine? 14:52
or only OUT operands?
nopaste?
clunker3 pasta.test-smoke.org/ or paste.husk.org/ or nopaste.snit.ch:8001/ or rafb.net/paste or poundperl.pastebin.com/ or paste.scsys.co.uk/
purl nopaste is at nopaste.snit.ch/ (ask TonyC for new channels) or rafb.net/paste or poundperl.pastebin.com/ or paste.scsys.co.uk/ or App::Nopaste or tools/dev/nopaste.pl or at www.extpaste.com/ or paste.scsys.co.uk (for #catalyst, #dbix-class, #moose and others)
nopaste "kjs" at 193.1.104.8 pasted "disallow IN operands before '='" (22 lines) at nopaste.snit.ch/14671
kj Coke: check out the nopaste
if you compile with this, then the make process won't finish, because apparently some libs that are compiled (to pbc I'd say) as part of the make process have cases with some misuse 14:55
specifically: runtime/parrot/library/Crow.pir:16 getopts = push 'type|t=s' 14:56
and indeed, I wouldn't write that like that: push is done on getopts (a PMC), so that's not an OUT operand 14:57
Coke kj: we need to disallow INOUT also. 14:58
can we change that line to != PARROT_ARGDIR_OUT ?
(does that do what I want?)
kj exactly 14:59
Coke SPIF.
kj then it should work, but when applying, some library code should be updated, as I mentioned above
Coke ayup. I'll make sure 'make test' works before committing, at least. 15:00
kj++ 15:01
ayup. I'll make sure 'make test' works before committing, at least.
er,
kj++
kj Coke++ # cleaning up tickets
you've been working hard the last 2 weeks!
Coke almost like I'm avoiding studying for a final. 15:02
15:02 rindolf joined
rindolf Hi all. 15:02
kj Coke: there's another deprecated thingy in IMCC, it's in DEPRECATED.pod, but not sure if there's a ticket. It's the functions iINDEX{SET,FETCH} 15:03
they make indexing strings work and convert them into substr with a slice size of 1. 15:04
around line 403
15:04 Andy joined
kj the parts with the 'S' test can be removed I think, but again, there might be some library code that uses this, which should be updated (but ISTR I did some of those conversions few months back) 15:05
Coke if you want to make patches for them (just against imcc.y and I can regen things), I can do the actual PIR cleanup.
kj I'm kinda working a bit now, but I can do that later today, otherwise tomorrow.
pmichaud I know that I've seen a few cases where people use indexing on strings.
might even be some in the rakudo libraries :-( 15:06
kj ah what, the heck. I'll make the patch now, it's easy enough (but then I should get back to work! :-)
Coke kj: slight bug in patch? 15:07
nopaste "coke" at 193.200.132.135 pasted "slightly modified" (35 lines) at nopaste.snit.ch/14672 15:08
Coke JSON = compreg "JSON"
1298:=item B<compreg>(out PMC, in STR) 15:09
1306:=item B<compreg>(in STR, invar PMC)
kj ah I know
that's what I was afraid of...
Coke (I also added in the name of the opcode in the error message)
kj I basically assumed that all variants have at least the same direction on the first operand
the trouble is, if you really want to have the opinfo object of the instruction that is actually used, then you need to know its full signature 15:10
Coke ok, but we can get that with the opinfo, neh?
kj now you just get the opinfo object of a random member of the opcode "family"
well, no.
now we apparently got the opinfo of the in STR, invar PMC variant
because we take the short name of the opcode 15:11
Coke right, but the func_ins call has to know what opcode is getting inserted, no?
kj and retreive the opcode of that
and use that as an index in the op_info_table
only when generating bytecode is the final opcode calculated
by generating _s, _p, _i etc on the short opname 15:12
so that's what I meant by "not easily fixed" as my first reaction :-(
so, the op_info_table has all the opinfo objects
and when doing a check whether a certain string is an opcode (like "print") it will return one member of the print_* family of opcodes 15:13
but you don't know which (it's probably the first one in the op_info_table)
so it can be print_s, print_i or any other
Coke where is that table defined? 15:14
kj so you might retrieve compreg_s_p, while you actually want compreg_p_p (or whatever)
op_info_table? I don't know
Coke core_ops.c ?
purl hmmm... core_ops.c is o.k.
kj never checked it. Check out the structure of the op_info object in include/op.h
s/object/struct/
so basically, the problem is, the check (of operand direction) MUST be done in the parser, because there you know whether '=' is used 15:15
but you don't know the full signature name yet. To calculate it in the parser is a bit of a waste of cycles
because it's already done in the backend (pbc.c i think)
so, of course you could change IMCC so that it will do it in the parser, but the phrase "good luck" would be applicable... 15:16
Coke do we have to make it a parse time error?
kj is there an option to make it a runtime error? 15:17
Coke if we know the direction when we're generating the bytecode..
kj you can't do the check anymore when doing bytecode generation, because we don't "remember" whether the user used PIR sugar ('='), as opposed to PASM style 15:18
Coke (we don't know then that we had the = syntax, though. we'd have to pass that along too, which smells evil.)
kj exactly
Coke so, good luck it is. hurm.
kj so, we can close the ticket in version 1.5 (or 2.0? I have to check ROADMAP)
(if s/imcc/pirc/ :-) 15:19
Coke is pirc going to make this any easier? 15:20
kj it's implemented in pirc ;-)
15:20 jhorwitz joined
kj pirc compiles well now on linux I think, so you could check it :-) 15:20
(but probably I only check for ARGDIR_IN, which should be ARGDIR_INOUT, but that's easily fixed) 15:21
jhorwitz crawls out from under a rock
Coke moves the rock and drops it back on jhorwitz.
kj Coke: I'll send you the substr patch, but you'll have to recreate it, as I don't properly invoke bison (i'm on windows+cygwin here) 15:22
Coke can you attach it to the ticket? 15:23
kj so I'll just send the imcc patch, and you can regenerate imcparser.c from that, is that ok?
sure
will do
particle WHO BROKE PARROT?
kj eh, just the imcc.y patch I mean
Coke yes. sending patches for -just- imcc.[yl] is preferred, I think.
jimmy its self
Coke particle: Could you be more specific?
rindolf In file included from src/pmc/default.c:17: 15:24
src/pmc/pmc_default.h:18: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PMC’
Should I run make clean first?
particle compilers\\imcc\\pcc.c(232) : error C2057: expected constant expression
compilers\\imcc\\pcc.c(232) : error C2466: cannot allocate an array of constant size 0
char bufcache[lenpref + lenitem * PCC_GET_ARGS_LIMIT + lensubf];
not c89-friendly
Coke rindolf: is that in the lex4 branch? 15:25
rindolf Coke: no, trunk
Coke realclean can't hurt. 15:26
particle fixes and rebuilds
kj Coke: the $S0[1] => substr X, 1, Y mapping doesn't seem to be in DEPRECATED 15:27
ISTR it was there..
pmichaud I don't remember the $S0[1] deprecated note, but I might've just missed it.
kj pmichaud: maybe it's no longer deprecated. I'm quite sure it was at some point. 15:28
15:30 apeiron joined
jimmy there were some errors in src/string.c? 15:30
pmichaud it'd be fine with me if it was deprecated. :-) 15:31
dalek r32978 | bernhard++ | trunk: 15:39
: [Pipp] Try to add support for class constants,
: but do not succeed yet.
diff: www.parrotvm.org/svn/parrot/revision?rev=32978
jimmy bernhard++ 15:40
barney I need to take a fresh look at $?INIT later 15:41
jimmy i think pipp will support thread. 15:55
expect it. 15:56
jonathan Not until Parrot 1.5...
...probably... 15:57
rindolf Coke: now it compiles.
jonathan starts some MMD and type related Rakudo hacking 16:01
pmichaud please don't say "not until Parrot 1.5" -- I think it's misleading. 16:02
some things might happen sooner than that.
jonathan OK, they *may*.
pmichaud no, I'm thinking in the other direction.
jonathan Like, they likely will?
pmichaud the point of the roadmap was to identify the _latest_ release in which something would be available, not the earliest.
jonathan Ah, true.
OK, by Parrot 1.5 at the latest. 16:03
cognominal compiling the last svn after a make realclean , and got an error in io.h:248
apparently PARROT_API is not know anymore
jonathan cognominal: Maybe try a make realclean? 16:04
Infinoid good morning. how's lex4 today?
jonathan If you didn't already.
pmichaud still working on lex4
I understand a bit more about continuations this morning.
kj cognominal: PARROT_API is now PARROT_EXPORT
Infinoid still refcounting woes? 16:05
pmichaud right now it appears there's a problem with :vtable
cognominal trying to remove the io.h, before doing the svn update. 16:06
pmichaud I think I'm going to add our refcount traces to trunk, so I can see what's happening there.
(by way of comparison)
pmichaud does that now. 16:07
Infinoid cool
cognominal apparently a glitch with svn...
pmichaud it looks to me as though contexts for PCCINVOKE are never reclaimed.
jonathan The bug I fixed last night was a discrepancy in PCCINVOKE stuff. It wouldn't surprise me if there were more there... 16:09
pmichaud actually, I need to revert that.
(based on my current understanding of retcontinuations)
16:09 workbench joined
pmichaud my current working theory is that retcontinuations do not refcount their from context 16:11
and that it's the ->ctx refcount in Sub that is causing such contexts to be prematurely recycled. 16:13
jonathan compilers\\imcc\\pcc.c(232) : error C2057: expected constant expression 16:15
(In trunk, latest revision)
compilers\\imcc\\pcc.c(232) : error C2466: cannot allocate an array of constant size 0
particle jonathan: yep, that's chromatic's fault 16:16
pmichaud 15:24 <particle> compilers\\imcc\\pcc.c(232) : error C2057: expected constant expression
particle i've been dealing with $job, so haven't fixed
pmichaud 15:24 <particle> compilers\\imcc\\pcc.c(232) : error C2466: cannot allocate an array of constant size 0
15:24 <particle> char bufcache[lenpref + lenitem * PCC_GET_ARGS_LIMIT + lensubf];
15:24 <particle> not c89-friendly
jonathan OK, not just windows...
pmichaud no, just c89
so probably just msvc
particle gcc--
pmichaud depends on what other compilers allow.
particle how the heck are we supposed to keep to c89 if most of the developers don't have a compiler that enforces it? 16:17
gcc--
jonathan By having Windows developers with compilers that don't suck? ;-)
Infinoid or, at least, suck differently enough to let us map out the intersection 16:18
jonathan Patched it. 16:22
dalek r32979 | jonathan++ | trunk:
: [imcc] Fix broken build for MSVC++. This may not be the ideal fix, but a better one doesn't come to mind right away.
diff: www.parrotvm.org/svn/parrot/revision?rev=32979
moritz karma MSVC
purl msvc has karma of 121
jonathan hah 16:23
karma C
purl c has karma of 7300
moritz karma C++ 16:24
purl c++ has karma of -80
Infinoid karma gcc
purl gcc has karma of -30
Infinoid poor gcc.
Coke part of the problem is that gcc may be warning about it, but no one is keeping on top of the warnings.
also, newer versions of gcc allow you to turn those into errors.
particle that warning should be fatal
Infinoid I check warnings on my platform once a week or so
Coke ... have you fixed them all, because I have a TON of them. =-) 16:25
particle but, gcc doesn't allow only c89 warnings to become fatal
Coke particle: newer versions do< I think.
particle it's all warnings or none
ok, that's news to me
Coke I had opened a ticket about it.
Infinoid Coke: how long have those warnings been there? things were looking pretty good last sunday, there were a few files that had known warnings that I can't fix, which have been there forever 16:26
Coke Infinoid: some time. 16:27
particle: I can't find my ticket. it was about gcc 4.mumble.
particle: -Werror=<specific wnaring> 16:28
gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
Infinoid: once I finish the tcl spec test run, I can see if I can clean any of them up.
particle coke: seems hints or warnings.pm needs an update 16:29
Coke there was, as I say, a ticket. ISTR I thought we needed to tie it to the config probe for gcc version. I also STR that someone else thought we could just shove it in. 16:30
particle depends on the min supported gcc i suppose 16:31
Coke I think that was in the ticket. =-)
rt.perl.org/rt3/Ticket/Display.html?id=50908 16:32
cotto said it was resolved.
it is possible that no one with a sufficient recent gcc tried to build between that commit and your build.
nopaste "Infinoid" at 96.238.213.50 pasted "warnings on x86-64" (26 lines) at nopaste.snit.ch/14673 16:33
Infinoid those have all been there for a while, and I think they all rely on fixing either system headers, bison/flex, or the headerizer.
Coke Infinoid: yah. be nice if we could disable warnings when run on certain files. 16:34
Infinoid I take it you're seeing more?
Coke I will have to check. 16:35
particle coke: we can, there's a cflags file for that
Coke particle: gcc on feather catches 'statement after declaration' as an error. 16:36
so if there are other gcc warnings that should be errors, we can add them.
particle i think you meant declaration after statement. good!
Coke yaya. 16:37
OH. 16:38
no, it doesn't fail. hurm.
16:38 rdice joined
Coke (I built one file that warned; the file was still created.) 16:38
pmichaud makes another stab at lex4 16:39
Infinoid current lex4 doesn't build for me, segfault in PGE, known issue or specific to my box? 16:40
moritz known
pmichaud known.
Coke particle: reopened ticket. 16:43
particle coke++ 16:47
Coke best part? ticket was still assigned to cotto. Muahahah. 16:48
Coke wonders how many tests would pass if he got bigint support working. 16:49
how do I check to see if a $N0 is NaN? 16:51
there doesn't seem to be an opcode for it. 16:52
Infinoid goes and figures out how to disable stack randomization, to make debugging easier. 16:53
Coke we are relying on isnan from math.h ? 16:54
:q 16:55
particle coke: EDOM and ERANGE in errno.h signify domain and range errors
EDOM signifies Inf and ERANGE NaN iirc
i'm not sure isnan is c89 16:57
it's worth probing for, if it's not
16:58 tomyan left
Coke so if I have a $P1 == Float, how do I check for nan-ness? 16:59
do I just -use- it and check the error?
particle you can check to see if it equals itself 17:01
Infinoid hmm. the problem with freeing the context is that the system just turns around and allocates it again. that doesn't help us find any dangling pointers to the old context, which are now pointing at the new one, or possibly pointing somewhere in the middle of something else.
pmichaud r32980 in lex4 passes all parrot tests and rakudo 'make test' on my system.
Coke particle: that's evil.
Infinoid I'm going to try catching this with efence.
particle isnan(Float x) { return x != x }
pmichaud and, afaict, it doesn't leak. 17:02
Coke particle: that's C. =-)
dalek r32980 | pmichaud++ | lex4:
: Return to the original Parrot_pop_context semantics
: (although I'm not entirely sure why it works).
: Also, Sub VTABLE_invoke creates retcontinuations by default
: for normal subs, but outer subs convert this to a normal
: continuation and set a sub->ctx reference.
diff: www.parrotvm.org/svn/parrot/revision?rev=32980
Infinoid tests that instead :) 17:03
particle coke: an isnan op may be desired, but yeah
jonathan pmichaud: Does that want testing on Win32 now? 17:04
pmichaud jonathan: yes.
jonathan pmichaud: OK, doing
dalek r32981 | pmichaud++ | lex4:
: Remove a spurious comment left over from debugging.
diff: www.parrotvm.org/svn/parrot/revision?rev=32981
pmichaud I'm heading for lunch -- bbi45
reports on lex4 very welcomed. 17:05
Infinoid I still get a dead context exception trying to build PGE
but at least it doesn't segfault.
pmichaud Infinoid: maybe make realclean, just to be sure?
Infinoid I did, doing it again just in case
pmichaud and make sure you don't have any local changes?
Infinoid I don't. 17:06
pmichaud okay./
TimToady you just need to invent undead contexts and then keep the silver bullets handy
Infinoid I have high hopes that efence will turn something up.
(turning the undead is acceptable, too.) 17:07
pmichaud I think it's very likely that this (or something very close to it) will be what merges back to trunk. It's very close to the original trunk code.
jonathan pmichaud: lex4 builds here 17:08
smokin'
Infinoid ok, maybe my problem is something 64-bit related. 17:09
dalek r32982 | jonathan++ | trunk:
: [rakudo] Refactor subtypes. The subset class that should never have been is gone, and we can now find out quickly what real, non-subset type was refined.
diff: www.parrotvm.org/svn/parrot/revision?rev=32982
Infinoid TimToady: aren't the undead another kind of memory leak? :) 17:10
pmichaud I think our problem is that the undead have been supporting us for a while, but they're not first-class citizens in our world.
-D80 gives some pretty interesting diagnostics now, also. 17:11
as in, it shows when we "force" recycling of a context in Parrot_pop_context. 17:12
jonathan pmichaud: Bad news - at least one fail.
(In make test)
pmichaud okay.
I'll get lunch, then return. 17:13
details on failures welcomed.
Infinoid: even if PGE doesn't build, what does "prove -r t/op" and "prove -r t/pmc" give? 17:14
rindolf Can anyone help with sial.org/pbot/33299 ? 17:15
nopaste "jonathan" at 85.216.157.73 pasted "lex4 on win32 make test" (7 lines) at nopaste.snit.ch/14674
jonathan rindolf: Ouch! That's...quite a big test case. :-) 17:16
rindolf jonathan: why isn't there a line number? 17:17
jonathan Because we haven't got the stuff in place to show one yet.
But that's failing quite far down...
Hmm.
Oh, no
Only in PAST;compiler
rindolf: Will need to narrow down exactly what causes the compilation error. 17:18
It's not making it to runtime.
nopaste "Infinoid" at 96.238.213.50 pasted "results of t/op and t/pmc on linux/x86-64" (227 lines) at nopaste.snit.ch/14675 17:20
Infinoid pmichaud: ^^
rindolf jonathan: it's the portion starting from my $start = int(@*ARGS.shift); 17:24
jonathan Ah. 17:26
I suspect that it's because we haven't got type coercions working yet.
(e.g. the int(...)
rindolf: Try it with my $start = +@^ARGS.shift; 17:27
erm
my $start = +@*ARGS.shift; 17:28
+ numifies it
And is implemented.
rindolf jonathan: no, it's my ($g_val, $composition) = Graham($n);
jonathan: list assignment.
purl list assignment is default in perl 6, but don't know about a 'default' in PAST. probably better to specify either list or scalar
Coke particle: your equality trick works in PIR, even.
it does seem to just be a trick, though. 17:29
jonathan rindolf: Aha.
Coke particle: what about inf?
jonathan List assignment is todo. Being done soonish, I believe.
rindolf jonathan: ah.
jonathan: OK, now I'm getting a few run-time errors. 17:34
Without line numbers. <sigh />
nopaste "jonathan" at 85.216.157.73 pasted "pmichaud - lex4 results for Rakudo spectest" (8 lines) at nopaste.snit.ch/14676
jonathan rindolf: Yes, there's some work to do before we can show those, I'm afraid. :-( 17:35
rindolf jonathan: OK.
dalek r32983 | jonathan++ | trunk: 17:42
: [rakudo] Pull refined real type out of subtypes when constructing a signature, so multiple dispatch handles subtypes more correctly (possibly even completely correctly now).
diff: www.parrotvm.org/svn/parrot/revision?rev=32983
Infinoid pmichaud: I hate to keep giving you bad news... but nopaste.snit.ch/14662 still leaks in r32981. 17:49
nopaste "Infinoid" at 96.238.213.50 pasted "valgrind leak summaries" (121 lines) at nopaste.snit.ch/14677
pmichaud Infinoid: ... that's odd. 17:55
I don't get at all the same results with valgrind.
oh wait, yes I do now.
okay, I'll look into that first. 17:56
another strategy I thought of at lunch is to have Parrot_free_context "poison" the freed contexts, but don't mem_sys_free them or place them on the recycle pool. Then if anything tries to use them after poisoning, we'll know what it is. 17:59
Coke if it's like GC, isn't the problem that something is freeing it, not that something is using a freed one?
Infinoid that's exactly the reason why I tried going the efence route. unfortunately, other issues prevented me from getting far enough to test that
pmichaud Coke: it's a bit of both. 18:00
Infinoid pmichaud: valgrind's --freelist-vol option looks useful for that.
pmichaud Coke: contexts use a refcount approach for management, instead of marking
so it's entirely possible for something to refer to an otherwise-freed context
18:01 chromatic joined, chromatic left, chromatic joined
Infinoid I've also tried doing an rwatch on the contents of the freed context 18:01
pmichaud sub/pmc/sub.pmc:284 18:03
/* XXX: Why is this reference needed...? */
Parrot_context_ref(interp, context);
I don't like those lines.
it bugs me that I have a Parrot_context_ref with no corresponding free. 18:04
dalek r32984 | jonathan++ | trunk:
: [rakudo] Another spectest that we now pass much of.
diff: www.parrotvm.org/svn/parrot/revision?rev=32984
pmichaud I'm going to try the context poisoning approach, I think. 18:05
is there a better value to use besides "NULL" to force a segfault if dereferenced? 18:08
particle 0xdeadbeef ?
0x5afecafe ?
pmichaud I know we use that to indicate freed stuff in gc; I'm not sure I want to re-use it for contexts.
(0xdeadbeef)
it might lead people to look at the wrong gc 18:09
particle it'd be nice to have a unique one
chromatic 0xbeefcafe 18:10
pmichaud okay.
that it is.
chromatic Because we can't spell 0xmmmbrisket
... without base-22 compilers.
Infinoid oh, great... PGE/builtins_gen.pir is built just fine when I run parrot under valgrind 18:15
18:15 sjansen joined 18:19 integral joined
Infinoid oh, this is interesting. building TGE, I caught Parrot_Continuation_mark calling mark_context on a freed context 18:21
jonathan Infinoid: Is it a RetContinuation or a Continuation?
(check vtable->base_type) 18:22
Infinoid that was under valgrind, I'll try gdb
Coke 0xdeadabba 18:24
Infinoid its a enum_class_RetContinuation
jonathan Hmm. A RetContinuation that didn't get invoked yet, then. 18:25
Infinoid well, according to valgrind, its context has been freed 18:26
when I ran it under gdb, the freed buffer was probably immediately reallocated, so it died a couple of calls later. 18:27
pmichaud that's why I like the poisoning approach (about finished)
it can prevent reallocation of contexts.
Infinoid yes, valgrind does that automatically, but your way would help the gdb case too 18:28
jonathan So are we giving ref counts to contexts when RetContinuation references them or not? 18:29
pmichaud no.
(for now.)
nopaste "Infinoid" at 96.238.213.50 pasted "here's what valgrind had to say." (96 lines) at nopaste.snit.ch/14678
pmichaud right now I'm pretty closely following what trunk does in this respect.
jonathan It seems in some cases the RetContinuation outlives whatever was the thing that did have the ref count.
Infinoid so that tells us where it was freed, and also where it was accessed
jonathan I guess you don't invoke the retcontinuation if you throw an exception and leave a sub that way, for example. 18:30
Infinoid isn't this an argument for the retcontinuation holding a reference on the context?
pmichaud I don't understand the whole retcontinuation approach yet. 18:31
Infinoid you guys understand it a lot better than I do. :)
jonathan If we have cases when a RetContinuation (correctly) outlives anything else holding a reference to a context, that'd suggest that yes.
pmichaud my working theory is that retcontinations are things that have no other references to them 18:32
Coke 0xdeafabba
Infinoid if not, then it just slows things down and doesn't provide a benefit (except, er, not crashing), right?
pmichaud and if something takes a reference, then the retcontinuation is supposed to be converted to a continuation
(and the reference count is increased)
Infinoid Coke: ouch.
jonathan pmichaud: OK, I think here the case here may be along the lines of, we never invoke the retcont, so it keeps a pointer to the context. 18:33
pmichaud jonathan: and the retcont has to still be a live.
jonathan But in the meantime, the other thing that did reference it has freed it.
pmichaud "alive"
oh. 18:34
Infinoid goes and reads docs/pmc/subs.pod
pmichaud jonathan: if what you're saying is correct, then that means something is taking a reference to a retcont that shouldn't be.
(to the context that a retcont is using)
and I still haven't completely figured out why it's the "from" pointer we keep track of and not the "to" pointer. 18:35
Infinoid cc->to_ctx is the one that got freed 18:36
pmichaud right. 18:38
Infinoid you're talking about the refcounting done in new_continuation(), right?
pmichaud in a variety of places. 18:39
invalidate_retc_context is another.
Infinoid is there an assumption that to_ctx is somewhere in the current context's caller_ctx chain, and is protected by that?
pmichaud there may be.
I do know that contexts were viewed as a stack for quite a long time. 18:40
so returning to a point higher in the stack was the same as "releasing" all of the contexts on top of it.
(I think.)
Infinoid which makes sense, coming from a C background
pmichaud (I know that it was viewed as a stack... I don't know that the release implementation was made that way.) 18:41
chromatic That sounds right to me. I kept thinking "That won't work; this is a call *graph*." 18:47
pmichaud a-ha. 18:48
this might be a problem:
if a RetContinuation is cloned, that produces a Continuation
the Continuation holds a reference to the context
if that Continuation is then destroyed, it releases its reference 18:49
jonathan pmichaud: I'm starting to think we really should just refcount RetContinuations.
pmichaud oh... never mind -- what I was thinking of is okay 18:50
jonathan Lying about not having a reference when we have one seems to be causing trouble.
pmichaud jonathan: (1) I've gone that approach, without too much success
Infinoid ...and relying on nothing else calling Parrot_free_context in the meantime (holding a reference or not)
dalek r32985 | kjs++ | trunk:
: [docs] update PIR articles.
diff: www.parrotvm.org/svn/parrot/revision?rev=32985
pmichaud (2) I'm still wondering _why_ we need RetContinuations in the first place
18:51 kj joined
jonathan pmichaud: I think the idea is that we have a kind of "one shot" continuation, that is cheap and doesn't need us to make a real continuation. 18:51
Just for the purpose of doing a return.
pmichaud I can't tell that it's any more/less cheaper than a normal continuation.
certainly that's not the case if we're refcounting them.
dalek r32986 | kjs++ | trunk: 18:52
: [pirc] tests, documentation and fixes.
diff: www.parrotvm.org/svn/parrot/revision?rev=32986
pmichaud (well, it may still be the case ... but I haven't seen it.)
jonathan Because you don't have to "capture" (wrong word...I know) the whole call chain, which I guess you'd need to do if you had a real continuation. Or something liek that.
*like
pmichaud real continuations don't really capture the whole call chain
jonathan I always had a hard time following those areas of the code, though.
pmichaud they do make sure that all of their callers are continuations 18:53
but that sounds like a circular argument.
again, it might've been the case that retcons were cheaper when we had a stack, but that's no longer the case. 18:54
or if it is, I haven't seen how.
tewk_ Yeah, continuations need to capture the whole call chain or at minimum mark contexts as COW. 18:58
pmichaud okay, here's the other piece that bothers me. Currently a sub that is an outer sub changes its retcontinuation into a full continuation 19:04
presumably this is because its context may need to live beyond the completion of the sub
but in other locations when we convert a retcontinuation into a full continuation, we also convert all of its callers as well (see invalidate_retc_context) 19:05
I'm thinking that Sub.invoke should call invalidate_retc_context instead of converting the retcont directly... 19:06
but...
in what situations are we going to have the case that everything doesn't just end up being a continuation ?
I suppose for deeply nested chains of non-outer subs we'd have retcontinuations
but I suspect that doesn't happen all that often in most of our target languages. 19:07
Coke why not try ripping out retcons and see what happens? 19:08
chromatic +1 here
19:08 barney joined
Coke chromatic can always speed things up later. :| 19:09
pmichaud that's a bit more significant a change than I was wanting for the branch (more)
I guess I'd also like to hear from allison if she things we need retcons, and why.
*thinks
okay, I'm curious what people get with r32987. I added context poisoning to the mix. 19:11
dalek r32987 | pmichaud++ | lex4: 19:12
: "Poison" contexts whenever they're released to the recycle pool.
: If anything then tries to use a poisoned context, we should get
: some sort of segfault or error. There's also a "slot = 0" line
: (commented out by default) that prevents contexts from ever
: being re-used but that doesn't show up as a memory leak in
: valgrind.
diff: www.parrotvm.org/svn/parrot/revision?rev=32987
pmichaud uncomment src/gc/register.c:605 to prevent contexts from being reused. Then anything that has a reference to a freed context will have a reference to a poisoned context (and hopefully we can find out what it is).
s/reference/pointer/g 19:13
I have to run a couple of errands -- back in 30 :-|
19:16 stockwellb joined
stockwellb init::manifest - Check MANIFEST...No such file: languages/perl6/src/classes/Subset.pir 19:16
oops I didn't no pasting would do that. I'm sorry.
Anyway. I got this a Configure error after I cleaned and svn updated. 19:17
jonathan oh, shite...manifest 19:19
barney fixed
Coke manifest? I hateses it. 19:20
stockwellb I didn't think that was worth a bug entry. Hope that's alright.
nopaste "chromatic" at 69.64.234.10 pasted "Coke: Naive MMD Caching; does this help Partcl?" (427 lines) at nopaste.snit.ch/14679
dalek r32988 | bernhard++ | trunk:
: Regenerating MANIFEST
diff: www.parrotvm.org/svn/parrot/revision?rev=32988
19:21 masak joined
stockwellb Is it safe to kill a make in progress? 19:21
Coke don't see why not.
barney famous last words 19:22
purl No, I am certain it takes a 7D bolt.
nopaste "particle" at 98.232.28.49 pasted "test timings for perl .t files with pasm_* functions" (74 lines) at nopaste.snit.ch/14680
particle stockwellb: some of those tests could use conversion to pir
those with the longest time would give the greatest benefit
the first three probably shouldn't be converted, though 19:23
stockwellb I'm going to run make with the timer on and see which is the biggest hog.
particle ok, i did an ack -al pasm_ 19:24
to get the list of files that had pasm in them
stockwellb Oh damn thats what the nopaste is...
particle yes, it's a few days old, though
stockwellb gc sounds scary, maybe string? 19:25
particle sure
stockwellb dang, my first assignment as a Parrot secret agent!
Infinoid is it a bad sign when you rip context refcounting out entirely, and suddenly parrot builds again, and passes all core tests? 19:26
particle check for leaks
chromatic Only if you like memory leaks!
tewk_ so subs today are more like blocks than subs right. 19:27
particle the basic hll block in parrot is a pir sub
Infinoid oh, I'm sure it will leak like crazy :)
when I get back, I'll see how hard it is to fix that up 19:28
lunch &
dalek r32989 | bernhard++ | trunk: 19:30
: [codingstd] satisfy trailing_spaces.t and c_parens.t
diff: www.parrotvm.org/svn/parrot/revision?rev=32989
r32990 | jonathan++ | trunk:
: [rakudo] Put something in to make traits on routines work for now. The whole traits thing needs a good refactor to eliminate the duplication here and elsewhere - will attend to that in the not too distant future.
diff: www.parrotvm.org/svn/parrot/revision?rev=32990
19:37 sjansen joined
kj Coke: I'm looking quickly into the disallowing of PASM registers in PIR mode 19:41
there's something weird going on. For instance, runtime/library/parrot/Digest/MD5.pir, line 157, an error message (after applying your patch in the rt queue) indicates that no PASM register is allowed. Pasm_file variable is 0, and the matched text is "I0". But that's stragne, because it's clearly $I10. I think it might have to do with the order of the rules in the .l file; I'll move it *after* the rules for $[SNIP]{DIGITS}... that should help. 19:43
dalek r32991 | pmichaud++ | lex4:
: I've convinced myself that this spurious increment of a context
: reference is wrong -- it causes context leaks when present, and
stockwellb is amazed at the total lines of code in string.t (over 2900). 19:44
dalek : there's no obvious place where it gets released. So, out it goes.
diff: www.parrotvm.org/svn/parrot/revision?rev=32991
19:44 chromatic joined
cotto pmichaud, ping 19:44
pmichaud pong
nopaste "cotto" at 96.26.202.243 pasted "almost-working grammar example" (77 lines) at nopaste.snit.ch/14681 19:45
cotto can you tell me what's wrong with that grammar?
jonathan chromatic: In the backscroll I think I read that you had written an MMD cache.
chromatic: But didn't see it ci'd... 19:46
cotto It's a simplification something I'm doing for pipp.
pmichaud cotto: what are you matching it against?
cotto example:
<x> foo </x> foo <y> foo </y>
pmichaud and what do you get when matching? 19:47
dalek r32992 | jonathan++ | trunk: 19:48
: [rakudo] Implement is default trait and use it as a final tie-breaker in multiple dispatch.
diff: www.parrotvm.org/svn/parrot/revision?rev=32992
jonathan needs nom - tests for is default afterwards
chromatic jonathan, it's a (messy) proof of concept that I nopasted for Coke to test.
nopaste.snit.ch/14679 19:49
jonathan Oh, I missed seeing the nopaste
chromatic It's very naive, but it's a start.
pmichaud cotto: I don't think that .return(0) means "fail"
nopaste "cotto" at 96.26.202.243 pasted "example of how the grammar doesn't work" (17 lines) at nopaste.snit.ch/14682
pmichaud PGE still implements an old version of S05 where returning a value (any value) meant something special -- but didn't necessarily mean "fail". 19:51
cotto ok. So how would I do match/fail?
19:51 gmansi joined
pmichaud if $S0 != $S1 goto fail 19:52
.return(1) # succeed
fail:
}}
<.fail>
hmmm. 19:53
maybe not.
jonathan chromatic: OK, not quite what I had in mind, but looks like it'd work.
pmichaud I would actually reverse the regex, though
check for '</x>' first, and then make sure that you were wanting to match a <x>
jonathan chromatic: I wanted really though to attatch the cache to the MultiSub (or some other place for the ops)
chromatic jonathan, I think we need a different approach for multi-vtables. 19:54
jonathan Since lexical multis or multis in anonymous classes won't last forever, but references to them would in a global cache like this, IIUC.
pmichaud of course, if you're certain that every open has a close, it could be done all in one rule with:
chromatic Hm, the MultiSub would be nice. I thought of that this morning, but I hadn't realized how nicely it makes cache invalidation.
jonathan Yes, that too. 19:55
I wasn't going for strings either.
I was going to chase down the type IDs of the parameters and cache on those.
Coke kj: that's why i didn't apply the patch. =-)
chromatic You'd be surprised how performance-sensitive this is. I used a naive sprintf to generate the type strings before, and it cost 5%.
pmichaud token { '<' (x) '>' <name>? '</' $1 '>' }
er, change that $1 to $0
jonathan Wow, I'd not have guessed that. 19:56
kj Coke: I did some moves in the lexer of rules, now make process finishes properly, but now the tests fail. maybe because .t files won't make the pasm_file variable set to 1
not sure, have to look into what it is.
jonathan I did also ponder trying to factor out the adding stuff to an MMD cache and so on.
So other MMD implementations can use it.
Perl6MultiSub won't always want to cache every result if computes, for example. Like if they're dependent on values than just types. 19:57
*it computes
Also, I'm confused why the name interp->gc_generation? 19:58
Were you pondering a more general mechanism other than MMD caching?
chromatic jonathan, I reused an unused interpreter slot because I didn't want to rebuild the whole system by changing interpreter.h
20:00 stockwellb left
cotto pmichaud, the actual matching I need to do is more complex, e.g. "<?" -> "?>", "<?=" -> "?>", "<script langauge='php'>" -> "</script>" 20:00
jonathan chromatic: lol!
:-)
chromatic There's another reason I didn't check it in. 20:01
jonathan OK
What shall we do?
Should I also prototype what I was thinking of too?
And then we can compare their performance wins/fails etc? 20:02
pmichaud cotto: okay. Anyway, .return(1) will currently mark a rule as succeeding (and abort the remainder of the match)
chromatic I think we need two different things. 20:03
Making op-to-vtable dispatch go through the marshalling/demarshalling is crazy.
dalek r32993 | kjs++ | trunk:
: [t] change a PASM register in a PIR sub into a PIR register. PASM register bad!
diff: www.parrotvm.org/svn/parrot/revision?rev=32993
cotto pmichaud, thanks. I'll see how far I can get with that. 20:04
chromatic We have fixed signature types at that point. Why throw them away by grabbing known-values out of registers, stuffing them in varargs, pulling them out, counting them, getting their types, and stuffing them into registers in a new context?
jonathan ouch!
OK, I need to eat. I'll spend some time looking at exactly what we're doing after I get back.
chromatic Adding a cache to MultiSub makes sense though.
jonathan Sure. We may have to keep them as two separate things, but we may be able to squeeze enough re-use out of it to not have to. 20:05
Anyway, bbiab 20:06
nopaste "pmichaud" at 72.181.176.220 pasted "I find this truly frightening..." (18 lines) at nopaste.snit.ch/14683
pmichaud okay, I'm less frightened. I didn't realize that Test::More had a 'like' function that depends on PGE 20:07
and since pge isn't currently building in the branch, that probably explains why it fails. 20:08
20:09 Lorn joined
cotto pmichaud, can a PIR macro be defined. then used in inline PIR chunks in a grammar? 20:10
dalek r32994 | kjs++ | trunk: 20:11
: [t] update PASM regs into PIR regs
diff: www.parrotvm.org/svn/parrot/revision?rev=32994
Coke pmichaud: that makes me think we need to think about our testing strategy.
pmichaud current lex4 branch passes all tests for me except those that are PGE based. 20:18
and think I know what's happening there
(PGE based or otherwise rely on PGE)
20:23 Lorn_ joined
bacek pmichaud: rakudo is slightly broken in lex4 20:24
is it expected?
pmichaud 20:18 <pmichaud> current lex4 branch passes all tests for me except those that are PGE based.
that would certainly include rakudo. 20:25
bacek pmichaud: ah, ok.
All tests successful.
purl ship that bitch
bacek Files=395, Tests=11643, 259 wallclock secs ( 3.41 usr 0.62 sys + 109.33 cusr 28.79 csys = 142.15 CPU)
Result: PASS
it's 'make test' in parrot. 20:26
looks good :)
pmichaud have to go pick up kids from school, back in 30 20:28
bacek: how about 32995 ? 20:29
(back in 30)
dalek r32995 | pmichaud++ | lex4:
: oops -- previous commit left poisoning commented out from earlier debug.
diff: www.parrotvm.org/svn/parrot/revision?rev=32995
bacek pmichaud: rebuilding... 20:30
pmichaud: ../../parrot ../../runtime/parrot/library/PGE/Perl6Grammar.pir --output=PGE/builtins_gen.pir PGE/builtins.pg 20:35
make[1]: *** [PGE.pbc] Segmentation fault
particle that's better 20:36
chromatic "better" 20:37
purl i heard "better" was *always* subjective.
particle thanks, purl
jonathan I get that on Win32 also. 20:38
nopaste "bacek" at 123.243.38.218 pasted "spoiled context segfault for pmichaud" (51 lines) at nopaste.snit.ch/14684 20:40
"bacek" at 123.243.38.218 pasted "result of bisect for pmichaud" (14 lines) at nopaste.snit.ch/14685 20:43
bacek "git bisect run make" rule the World! :)
particle bacek: it's failing *on purpose* 20:44
bacek afk # breakfast
particle: yes, I understand.
particle so bisect isn't helpful
Coke who is tak? 20:49
purl hmmm... tak is yes, nye is no
pmichaud segfault is occuring during load_bytecode 21:00
of P6object.pbc
that's early.
Coke that sounds familiar. =-)
pmichaud looks like it's occuring upon return from that. 21:01
chromatic . o O { Don't say IMCC. Don't say IMCC, } 21:04
Coke IMCC! 21:05
chromatic That's it, I'm checking in code to make Partcl *slower*. 21:07
Coke ... how will I noticeE? 21:08
chromatic Did you try that MMD patch? 21:09
Coke I ended up doing something about 2.5 hours in that broke the spec test run. 21:11
5% of 3.N hours is about 10 minutes. 21:12
so it's not going to help much. =-)
I'll run it overnight.
chromatic That patch should give you 40%. 21:13
Coke you say patch; is it not committed to trunk?
I wasn't seeing a 40% speedup.
chromatic Yes.
I nopasted it for you.
Coke lost in the mists of time. can you do so again?
21:14 davidfetter joined
chromatic nopaste.snit.ch/14679 21:14
Coke we have a STREQ macro, btw. 21:15
particle skoold! 21:16
chromatic I thought it was STR_EQ
No wonder I didn't find it.
Tene We could just s/_// everything in parrot. 21:17
Coke our choices of when to use and not use hyphens was made by a dyslexic ferret.
... and underscores are worse.
Infinoid: "is declared static and never used..."
(3*60*60*0.4)/60 21:18
purl 72
masak dyslexic ferrets seldom get appreciation for their work.
Coke (3.5*60*60*0.4)/60 21:20
purl 84
Coke chromatic: I've started the test run with a fresh parrot with your patch and a fresh tcl. I'll let you know what I find. 21:23
pmichaud a-ha!
src/inter_run.c:204 21:24
er, 304
ctx = runops_args(interp, sub, PMCNULL, NULL, sig, args);
...I bet that by the time runops_args is done, that context is already freed.
chromatic Makes sense why you'd see it from load_bytecode. 21:25
jhorwitz masak: ping 21:27
masak jhorwitz: pong
good evening, sir.
jhorwitz evenin'. :) was poking around november's CGI.pm last week. would you be interested in (eventually) helping out with a shiny new mod_perl6-aware CGI.pm? 21:28
masak jhorwitz: sure, why not? 21:29
sounds interesting.
I still intend for CGI.pm to go away eventually, though.
say, on a three-month time scale.
jhorwitz excellent! i'd love to have a bare-bones version w/o all the HTML crap.
(from the p5 version)
masak aye.
that's the idea. 21:30
pmichaud amen to that.
jhorwitz november works great under mod_perl6, but lots of tweaks to CGI.pm to make that happen.
masak jhorwitz: I suppose you've read Juerd's two emails about it from long ago.
jhorwitz hm, maybe, maybe not.
masak jhorwitz: I'd be very happy to collaborate.
jhorwitz masak++
masak jhorwitz: let me see if I can find the links.
Tene masak: I've been wanting to work on that for a long time
Juerd's posts. 21:31
There are copies in the pugs tree, I think.
masak Tene: I applied for a Perl 6 microgrant, but didn't get it
chromatic pmichaud, increment the refcount in src/inter_run.c:238?
masak Tene: will probably apply for a Perl microgrant in December.
pmichaud chromatic: exactly what I was looking at. :-)
masak Tene: maybe we should apply together. 21:32
pmichaud just got there.
masak jhorwitz: ah, here: groups.google.com/group/perl.perl6....ddfadad19b
jhorwitz clicks
pmichaud we then have to be sure to free the context after each call to runops_args
Coke chromatic: -feels- faster, anyway. 21:33
pmichaud or have the setval's do it :-(
Tene masak: sure. sounds great to me. 21:34
masak Tene: great. I'll make up a draft and get back to you. 21:35
jhorwitz masak: i agree with most of that
a bit shaky on the Web::Toolkit bit, but i haven't really thought about it much 21:36
masak jhorwitz: goodie. there will have to be _lots_ of discussion about the details when we create the new Web.pm module
or whatever it's to be called
jhorwitz *this*one should be CGI, as it will implement CGI, and only CGI. but like Juerd suggests, it should probably live under a namespace. 21:37
masak that can be arranged 21:38
jhorwitz or instead of a mod_perl6-aware CGI.pm, a common front end that's aware of both. 21:39
Coke chromatic: going to have to kill this and run it later; one of the kids is chewing up 80% of my CPU with his flash games.
jhorwitz smells smoke from his brain working hard 21:40
masak jhorwitz: are you on the November mailing list? would be good to have this type of discussion there. 21:41
jhorwitz URL?
masak hold on.
groups.google.com/group/november-wiki
jhorwitz danke 21:42
nopaste "pmichaud" at 72.181.176.220 pasted "This really looks like it deserves a refactor (src/inter_run.c)" (201 lines) at nopaste.snit.ch/14686
chromatic I'd like to see most of those go away, actually. 21:43
pmichaud well, me too.
but not this patch.
jhorwitz masak: joined. i'm planning on doing a lot of mod_perl6 work this weekend, so i'll try to start up a discussion while it's fresh in my mind. 21:47
pmichaud okay, another segfault. Let's see where this one is.
masak jhorwitz: excellent.
jhorwitz: I'll probably be doing quite a bit of November hacking this weeken.
s/weeken/weekend/ 21:48
the only cap is put by the studying I have to get done
jhorwitz studying-- 21:49
masak yes, but (knowing stuff)++
jhorwitz :) 21:50
pmichaud hey, PGE built! 21:52
now TGE fails. :-(
moritz "water bed theory of parrot building" 21:53
pmichaud sounds like the title of my next use.perl post.
moritz feel free ;) 21:54
22:02 apeiron joined
dalek r32996 | pmichaud++ | lex4: 22:03
: runops_args() returns a context to its caller (for return
: values), but the context it returns has often been
: reclaimed. So, we add a refcount to it and have all of
: its callers release the context.
diff: www.parrotvm.org/svn/parrot/revision?rev=32996
r32997 | pmichaud++ | lex4:
: If we clone a Coroutine, we need to add a refcount
: for its ->ctx reference as well.
diff: www.parrotvm.org/svn/parrot/revision?rev=32997
22:10 Hadi joined 22:14 Hadi left
pmichaud okay, now the problem appears to be that a coroutine is holding onto a context that has a caller_ctx that has been freed. 22:19
jhorwitz wonders what other phantom bugs r23996 has fixed
jonathan jhorwitz: I don't know, but I'm sure I've seen issues around that area... :-| 22:20
pmichaud well, we always did have some oddities with load_bytecode from time to time, so perhaps this is why. 22:21
jonathan . o O ( My word, no wonder MMD on ops got slow... )
pmichaud so, I guess I need to refcount the caller_ctx pointers as well?
jonathan If something holds a pointer to a context, it would seem to me that this should increment the refcount. 22:22
pmichaud it doesn't do so in trunk. :-(
but yes, I guess that's what will have to happen.
jonathan :-(
Yeah, but I don't know that trunk is a shining example of getting context ref counting right... 22:23
pmichaud fortunately it doesn't look like there are too many of them.
this will all be much easier when context's participate in the normal gc
jonathan If we can take any performance hit arising from it...
pmichaud well, that's why we need a better gc first
jonathan is specifically doing the MMD cache to avoid creating any more GCable objects.
pmichaud but even if we don't put the contexts into the standard gc, we'll start to see a performance hit as contexts live longer than they did before. 22:24
(as well as the ones they point to)
chromatic I don't think that'll really be a hit.
pmichaud and unlike the standard gc mechanism, there's not really a mechanism to say "we've already marked this context on this pass -- no need to mark it or its children again"
so, if a context marks its caller_ctx and its outer_ctx, and they're both the same, we end up doing the work twice. 22:25
jonathan chromatic: I think I'm going to hit the cache first right in the *ops* to avoid the marshalling. Think I'll be forgiven for this? ;-)
pmichaud (and outer_ctx == caller_ctx is common, which is why I made a special case for it.)
anyway, let's see what happens if we refcount caller_ctx 22:26
chromatic jonathan, are you concentrating on the vtable cache right now?
jonathan chromatic: I'm doing something that is generic enough we'll be able to use for vtables, as well as from multisubs. 22:27
chromatic Very nice. 22:28
jonathan If I can make it work. ;-)
jonathan wishes the names of the typed memory allocation macros would stay in his head 22:29
chromatic: Is there a mem_allocate_n_typed that promises zeroed? 22:31
Or does that one promise that?
Oh, don't need it here...but curious anyway...
chromatic mem_allocate_n_zeroed_typed, I believe. 22:33
I usually vi -t mem_allocate_typed and read the header.
jonathan OK
Tene ^]++ 22:34
particle: progress on reimbursements?
22:37 bacek joined, Whiteknight joined 22:45 apeiron joined 23:02 Limbic_Region joined
Coke restarts the tcl spec test. 23:02
chromatic: just kicked off the spec test again. 23:03
Tene: Oh, I'd definitely hit up the -treasurer- for that! 23:04
23:11 davidfetter joined
Coke upgrades his iphone. 23:15
chromatic: I appear to be hung on a test; wish I had finer-grained timings so I could know if this one test was slower. 23:16
(my parrot is up to 1.2G...)
23:23 Hadi joined
pmichaud first attempt at refcounting caller_ctx: FAIL 23:23
will try again a bit later.
23:24 Hadi left
TimToady pmichaud: have you got the svnadmin stuff yet? I just did some spec changes... 23:24
pmichaud yes, I did. But I'll repatch.
(I did get the svnadmin stuff)
I'll see about moving it to pugscode late tonight or early tomorrow -- any patches you make I'll do individually once I get things moved. 23:25
(not hard to manage, of course. There's this little handy utility called "patch" that I use... :-)
masak are all the synopses going to the Pugs repo?
pmichaud that's the current plan. 23:26
TimToady I've heard of it...
masak TimToady: you should try it, it's quite useful :)
TimToady I hear it's obslete now that we have revision control systems though...
masak if so, then I'm using the revision control systems the wrong way. 23:27
chromatic Hm, someone should interview TimToady about writing patch. 23:28
OH WAIT
masak :) 23:30
23:31 apeiron joined
jonathan OK, I just did a dumb benchmark after getting Perl6MultiSub using the MMD cache I've hacked up. 23:35
Runs 3 times faster with the cache.
masak "3 times faster" is the kind of speed we like around here.
TimToady but does it run 3 times correcter? :) 23:39
jonathan TimToady: In this case, only 1 times as correct. Damm. 23:40
masak better than nothing :P 23:41
23:53 tak joined
jonathan Damm. The way these MMD'd ops have been done makes it a pain to refactor it to work efficiently! 23:56
23:59 kid51 joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!