Parrot 1.2.0 released | parrot.org/ | 303 RTs left | Weekly Priority: Profiling
Set by moderator on 28 May 2009.
00:04 kesselhaus joined 00:05 kid51 joined
dalek cnum-dynpmcs: r71 | darbelo++ | trunk/src/pmc/decnum.pmc:
Add pow, pow_int and pow_float VTABLEs.
00:16
bacek purl: stupid girl 00:21
purl bacek: sorry...
bacek purl forget libraries
purl bacek: I forgot libraries
Whiteknight watching chromatic++ patching leaks is better then watching any TV 00:25
dalek TT #720 closed by jkeenan++: [PATCH] Typos in pdd19_pir.pod
Whiteknight well, maybe not as good as watching Lost, but he's only one man 00:26
bacek I hope he will not lost 00:27
dalek rrot: r39244 | jkeenan++ | trunk/docs/pdds/pdd19_pir.pod:
Correct POD errors, including those described by Klaus in
00:29
Whiteknight darbelo is kicking ass 00:33
darbelo++
darbelo Heh. Actually it was apretty slow day coding-wise. 00:34
Most VTABLE have adirect mapping to a decNumber function. They're pretty easy to do. 00:36
00:36 particle2 joined 00:46 whoppix joined 00:49 contingencyplan joined
dalek rrot: r39245 | jkeenan++ | trunk/src/oo.c:
Make file to c_parens coding standard.
00:49
kid51 Does anyone know how to fix failures in t/codingstd/c_arg_assert.t? 00:50
We've got 9 failures in this file: include/parrot/runcore_api.h
bacek kid51: make headerizer?
kid51 msg chromatic include/parrot/runcore_api.h has errors in t/codingstd/c_arg_assert.t 00:51
purl Message for chromatic stored.
00:51 eternaleye joined
dalek rrot: r39246 | jkeenan++ | trunk/lib/Parrot/Manifest.pm:
Make file pass perlcritic by adding coda.
00:56
01:09 whoppix joined
dalek rrot: r39247 | chromatic++ | trunk/src/interp/inter_create.c:
[src] Cleaned up the dynops destruction invocation code so that prototypes
01:09
rrot: r39248 | chromatic++ | trunk/src/runcore/main.c:
[src] Annotated runcore functions to pass c_arg_assert.t coding standards test.
01:10 wayland76 joined 01:11 amoc joined
bacek here will be dragons. 01:15
dalek rrot: r39249 | bacek++ | trunk/src/pmc/scalar.pmc:
[pmc] Scalar.bitwise_xor isn't MULTI.
01:16
rrot: r39250 | bacek++ | trunk/src/pmc/bignum.pmc:
[pmc] Cleanup BigNum's float handling a bit.
rrot: r39251 | bacek++ | trunk/t/pmc/multidispatch.t:
[t] Mark t/pmc/multidispatch tests with TODO TT#452 for future review and remove.
rrot: r39252 | bacek++ | trunk/src/pmc/bigint.pmc:
[pmc][cage] Fix BigInt pmc_reuse calls. We can't reuse Null.
rrot: r39253 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
Initial version of switch dispatch for MULTI

for basic (arithmetic) functions which uses only core PMCs. E.g. examples/benchmark/primes2.pir for 5000 elements executed in ~10 seconds instead of ~150 seconds.
01:20
rrot: r39254 | bacek++ | trunk/t/dynpmc/foo.t:
[t] Mark dynpmc failing test.
bacek msg chromatic can you review r39253 please?
purl Message for chromatic stored.
bacek afk # will be back in few hours 01:27
01:35 uniejo joined
dalek rrot: r39255 | whiteknight++ | branches/io_rewiring/src (3 files):
[io_rewiring] convert the Parrot_io_reads function
01:40
afk_Coke does a partcl build to see if anyone's been causin' trouble. 01:42
dalek rrot: r39256 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
[pmc2c] Fallback to MMD for non-core PMCs.
01:44
01:46 bacek joined
Coke bacek: I think you broke partcl. 01:46
bacek Coke: It works. After r39257. 01:47
make test still running.
Coke I just updated to r...4 01:48
reupdating...
bacek r..7 was a fix for dynpmcs.
dalek rrot: r39257 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
[pmc2c] Fix handling of dynpmc in switch-based VTABLE. Fix pcc signatures for fallback.
rrot: r39258 | bacek++ | trunk/t/dynpmc/foo.t:
[t] Remove passing todo in dynpmc/foo.
bacek here. see! :)
now definitely have to go. 01:49
Feel free to comment invocation of gen_switch_vtable in PMCEmitter.pm if it still broken.
Coke hurm. again, feels faster. 01:51
need to stat timing more things. =-)
I am assuming I'm not freeing some references to some PMCs. is there a util to show me where all the references are? 02:00
02:02 dukeleto joined
Coke hurm. I'm not running out of memory quite so soon, it would seem. 02:04
dalek rtcl: r395 | coke++ | trunk/config/makefiles/root.in:
add a shortcut for checkout out the entire tcl repo from CVS.
02:15
Coke I read my commit messages and sends and am horrified by my grammar. 02:16
kid51 quits to do Safari upgrade 02:24
dalek rtcl: r396 | coke++ | trunk:
ignore generated dir
02:39
rtcl: r397 | coke++ | trunk:
ignore correct generated dir
rtcl: r398 | coke++ | trunk:
ignore generated file
rtcl: r399 | coke++ | trunk/README:
hey, we can build now.
02:41 janus joined 02:43 whoppix joined 02:53 mikehh_ joined, tetragon_ joined
pmichaud fwiw, Rakudo seems to have gotten faster since r39239 03:01
(which itself was a significant speed improvement over r39238)
as of r39239, rakudo "make spectest" was 28m45. Now it's 26m36 03:04
dalek kudo: 7d75524 | pmichaud++ | build/PARROT_REVISION:
Bump PARROT_REVISION to get some more speed improvements.
03:05
03:08 ZeroForce joined 03:29 ZeroForce left, ZeroForce joined 03:31 ZeroForce joined 03:34 wayland76 joined 03:57 Theory joined
dalek kudo: 764684b | pmichaud++ | src/builtins/op.pir:
Fix cross-meta for user-defined infix ops.
04:02
04:29 darbelo joined 04:52 darbelo joined 04:57 tetragon joined 05:01 snarkyboojum joined 05:34 wayland76 joined 06:11 Theory joined 06:39 cognominal joined 06:47 wayland76 joined 06:58 muixirt joined 07:19 iblechbot joined 08:45 david joined 09:13 snarkyboojum joined
snarkyboojum hi all, I've downloaded and built parrot '1.2.0-devel built for nojit' on OS X and am trying to 'make' rakudo, but it's dying in the first step 'make: *** [perl6_s1.pbc] Segmentation fault' - how do I begin to debug? 09:16
any ideas?
09:21 ZuLuuuuuu joined 09:29 pjcj joined
cotto Parrot doesn't come with Rakudo. Are you using --gen-parrot on Rakudo Configure.pl or are you working with an installed Parrot? 09:29
also, where did you get Parrot from? Rakudo doesn't currently work against an installed Parrot unless the original source is there too. 09:30
Your best bet it to down a Rakudo release (or get it from git) and run perl Configure.pl --gen-parrot. That'll download and build a known-good version of Parrot for you. 09:33
snarkyboojum yep - using --gen-parrot on Rakudo Configure.pl 09:34
grabbed the latest tar.gz from github.com/rakudo/rakudo/downloads 09:35
so parrot builds ok, i.e. running perl Configure.pl --gen-parrot, but running make to build Rakudo fails with above message 09:36
cotto ok. You don't appear to be doing anything unusual. 09:37
snarkyboojum e.g. running ~/rakudo-2009-05/parrot/parrot -V gives me 'This is Parrot version 1.2.0-devel built for nojit. ...' etc
cotto Is there any way there's something left over from a previous build? 09:38
snarkyboojum I have a previous parrot installed from source I think
-- /usr/local/bin/parrot
cotto That shouldn't be messing anything up.
snarkyboojum which is a 'This is parrot version 0.5.0-devel (r23586)' 09:39
cotto *shouldn't*
snarkyboojum yeah..
cotto I'd ditch that anyway. That's ancient.
snarkyboojum yeah :)
cotto I saw a similar error on my machine (jaunty x86) when I was building Rakudo with an installed Parrot but had modified the source without installing the changes. 09:41
snarkyboojum googled it and found some references to that error on Linux boxes, but didn't get anywhere with a solution 09:42
was wondering how I'd start debugging such a problem
it's definitely 'parrot -o perl6_s1.pbc perl6.pir' which is failing 09:43
but no core dumps.. etc
cotto First, I'd get rid of that dusty old Parrot, run make realclean and rebuild Parrot and Rakudo. If the problem persisted, I'd use gdb.
snarkyboojum ok
not even sure what to hook gdb into 09:44
cotto Have you used gdb before and do you have it installed?
snarkyboojum used it to debug C programs
it's installed on my OS X machine yeah
cotto If it explodes again, feel free to nopaste a backtrace. 09:45
snarkyboojum cheers, will check it out a bit later
thanks
cotto ok. I'll probably be alseep by then, but it's not hard to find someone helpful here. 09:46
09:50 Austin_Hastings joined 09:54 wayland76 joined
dalek rrot: r39259 | fperrad++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
[codingstd] remove trailing space
10:08
snarkyboojum hi cotto - you still around? 10:17
cotto yup 10:18
snarkyboojum I'm attempted to use gdb to get a backtrace
haven't cleaned the old parrot yet
I'm basically done a 'gdb parrot' and then done 'run -o perl6_s1.pbc perl6.pir' 10:20
not sure if that's the correct way to do things :D
cotto that's what I'd do
snarkyboojum ok - and I get this error
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x72704620
0x72704620 in ?? ()
oops - apologies for multiline spam 10:21
can I post a backtrace here?
cotto we prefer nopaste
nopaste?
clunker3 pasta.test-smoke.org/ or paste.husk.org/ or nopaste.snit.ch:8001/ or rafb.net/paste or poundperl.pastebin.com/ or paste.scsys.co.uk/
purl i heard nopaste was at nopaste.snit.ch/ (ask TonyC for new channels) or poundperl.pastebin.com/ or paste.scsys.co.uk/ or App::Nopaste or tools/dev/nopaste.pl or at www.extpaste.com/ or paste.scsys.co.uk (for #catalyst, #dbix-class, #moose and others) or gist.github.com/
snarkyboojum ok
eek - which one is appropriate for here? 10:22
Austin_Hastings Whichever.
purl MAKE A DECISION
cotto any works. You can also use tools/dev/nopaste.pl, which is nice if you've got your output in a file 10:23
Austin_Hastings You paste to the site, it gives you a temporary url, you paste the temp url here.
snarkyboojum poundperl.pastebin.com/d3c30f64e
will that work?
Austin_Hastings I see a stack tract
* trace
snarkyboojum coolio
there you have it
Austin_Hastings Looks like it goes off the rails in mmd. 10:24
Were you just running standard code, or have you changed anything?
snarkyboojum multidispatch.c 711 :) nfi
standard yeah
essentially d/l cloud.github.com/downloads/rakudo/r...-05.tar.gz and followed the README on OS X 10:25
Austin_Hastings Okay. I don't know what the protocol is for this. 10:26
cotto I don't immediately know what the problem is. I'd recommend trying to catch someone more familiar with OSX.
Coke might be able to help. 10:27
snarkyboojum okydoke 10:28
any bug tracking type place I can put a dump?
Austin_Hastings trac.parrot 10:30
snarkyboojum ok - thanks again 10:33
dalek rrot: r39260 | fperrad++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
[codingstd] fix perlcritic
10:34
10:50 mikehh joined
Infinoid good morning 11:03
purl Here I am, brain the size of a planet, and all they say is 'Good Morning'
david That address looks suspect: 0x72704620. Each byte is ascii ('rpF ' in big endian, ' Fpr' in little endian). Not sure that means anything to anyone... 11:12
Infinoid msg whiteknight Hmm. src/io/io_private.h does have some flags for file/pipe/socket; but of course the interface to access those (Parrot_io_set_flags/Parrot_io_get_flags) is specific to FileHandles right now 11:20
purl Message for whiteknight stored.
Infinoid msg whiteknight Oh, there's a console bit too. That's interesting to me, there's a whole bunch of tty access crap (ioctls and functions and constants and such) that we can foist off into a subclass and keep the core clean 11:24
purl Message for whiteknight stored.
11:26 iblechbot joined 11:28 skids joined 12:47 bacek joined, Whiteknight joined 12:48 bacek joined
bacek hi there 12:50
purl niihau, bacek.
nopaste "Infinoid" at 24.182.55.77 pasted "packfile test failures in io_rewiring branch: apparently the PBC thaw code doesn't like a "type" value of 0" (35 lines) at nopaste.snit.ch/16727 12:53
Infinoid (whatever that means.)
jonathan privet, bacek # can't be bothered to go find rusky keyboard ;-) 12:54
Infinoid: Half the time, those disappear after a make realclean.
bacek jonathan: Гобрый вечер :)
No complains about my "switch-based" VTABLE for MULTI? 12:55
jonathan вечер? Tam mozet byt. ;-)
Infinoid jonathan: Interesting. Any idea what the mechanism behind it is?
(they don't disappear here, some branch change caused it)
jonathan Infinoid: Normally it's PBC compability breaks that cause it. 12:56
Infinoid: It's possible to cause it by screwing up a freeze/thaw routine though.
Or visit.
Infinoid well, we've been screwing with various I/O-related PMCs 12:57
and the lower level functions in src/io/
There are also some tweaks to not use PCCINVOKE as often
Infinoid tries dcommit on a branch for the first time 12:58
jonathan Infinoid: Which test fails?
Infinoid t/pmc/packfile* 12:59
jonathan It's possible that the I/O code is broken enough to have read the Wrong Thing but I suspect you'd have seen much more serious issues by now.
Ah.
Infinoid the "_pbc()" helper function fails to thaw a PBC file into memory for testing, so all the tests relying on that fail
jonathan Is that PBC file something you've generated? 13:00
Or a pre-existing?
Infinoid it's pre-existing. t/native_pbc/number_1.pbc 13:01
Maybe we just need to regenerate that
jonathan Oh yes
You've been re-organizing PMCs?
Removing and adding some?
In that case you will have broken PBC compatibility, should add an entry to PBC_COMPAT and should regenerate those.
Infinoid yeah, mostly adding 13:02
ok, thanks
jonathan This will bump up the bytecode version.
Way still want a make realclean after that (as will everyone post-branch-merge probably)
Infinoid realclean is a good policy anyway 13:03
jonathan aye :-)
dalek rrot: r39261 | Infinoid++ | branches/io_rewiring (6 files):
[io] Create an abstract close_piohandle() function in the main I/O API, so the various PMCs' destroy() vtables can call that directly.
rrot: r39262 | Infinoid++ | branches/io_rewiring (4 files):
[io] Make PIO_INVALID_HANDLE available to more than just the FileHandle PMC.
rrot: r39263 | Infinoid++ | branches/io_rewiring (3 files):
[io] Add a pipe creation function. (This needs implementation on win32; I've only handled unix so far.)
rrot: r39264 | Infinoid++ | branches/io_rewiring/src/pmc (3 files):
[io] Minor cleanups:

  * The Socket POD had some cutpaste errors from FileHandle.
rrot: r39265 | Infinoid++ | branches/io_rewiring/src/pmc/pipehandle.pmc:
[io] Create a PipeHandle PMC representing a pipe/fifo endpoint.
rrot: r39266 | Infinoid++ | branches/io_rewiring/src/pmc/pipe.pmc:
[io] Add the Pipe PMC.
rrot: r39267 | Infinoid++ | branches/io_rewiring (2 files):
[io] Implement Parrot_io_close_piohandle() and PIO_PIPE() on win32, too.
Whiteknight thanks jonathan++ 13:05
jonathan :-)
jonathan goes back to his pile of Slovak emails
Whiteknight I sort of figured the answer was something like that, but I haven't bumped PBC_COMPAT yet
Infinoid++ !! 13:06
This IO work is going to be a big improvemet, I think 13:08
Infinoid should probably write some pipe tests 13:09
Whiteknight Once we get pipes working, we could probably rewrite parts of the test harness to use them, to avoid relying on Perl5 13:10
Parrot calling Parrot, what a thing!!
Infinoid strange concept, indeed 13:11
bacek passing pipe with good tobacco to Infinoid. Test it, it's really good :)
Whiteknight: check pmc_pct branch. Parrot building Parrot :)
Whiteknight bacek++ quite impressive 13:14
I'm about to get on a plane here, so I can't check too mch 13:15
and I want to save some battery for the flight so I can parrothack midflight
Infinoid have a safe trip
Whiteknight bacek: How close is the pmc_pct branch to being merged to trunk?
thanks!
I'm actually going to disappear now. I'll talk to you guys later
bacek Whiteknight: it can be merged right now. 13:16
It's just not very useful ATM.
Whiteknight okay, thanks
13:29 kid51 joined
bacek Infinoid: around? 13:37
dalek rrot: r39268 | bacek++ | trunk/src/pmc/integer.pmc:
[pmc] Reuse "dest" pmc in Integer arithmetics when possible.

30% faster for calculating 5000 primes.
13:39
13:42 burmas joined 13:44 jsut joined
Infinoid bacek: yeah, what's up? 13:45
13:45 burmas left
bacek Infinoid: I'm going to make "reuse_or_new" from r39268 as new public functions. Any objections> 13:46
?
Infinoid I think the semantics are a little weird... in one case (reuse) other references to the same PMC will also be affected, in the other case (new) they won't be 13:48
bacek Not quite true. It's useful for 3-args ops in PMCs. 13:50
For example currently most PMCs will fail on "add", when will try to add to "ro" pmc.
It's just helper function to check corner cases 13:51
Infinoid true. (then again, "ro" support seems to be really sketchy in parrot; I honestly don't even know how to create one of those.)
actually I like what you've done in that r39268 patch, will reduce GC churn from creating new objects
bacek t/pmc/ro.t 13:52
purl t/pmc/ro.t is failing for me. It was passing for me last night
bacek purl: forget t/pmc/ro.t
purl bacek: I forgot t/pmc/ro.t
bacek Infinoid: I actually want it "public" because I want to use it in all PMCs. 13:53
Infinoid Making the function global probably wouldn't hurt. It also wouldn't hurt to ask the list how useful it is 13:54
bacek without copy-pasting every time...
Infinoid It will need some nice POD of course, so the callers can understand the side effects :)
bacek It is useful :)
But I'll fail in "nice POD". English in forth language, remember? :) 13:55
Infinoid maybe you'd be more comfortable trying Rude English or Very Very Rude English :)
bacek "We are f**cing going to kill dest. Try to preserve some guts" 13:57
jonathan bacek++
(That's only Rude English though. ;-)) 13:58
bacek jonathan: there is always way to improve something. Even my Rude English :)
jonathan remembers the time a Ukrainian guy tried to teach him that "жопа" means "heart" 13:59
bacek ROTFL 14:00
Parrot_pmc_try_reuse or Parrot_pmc_reuse_or_new? 14:01
any better names? 14:02
jonathan prefers the first.
14:03 Andy joined
bacek deal 14:04
Infinoid Does rakudo use parrot threads for much, yet? 14:05
bacek Infinoid: not yet.
jonathan Infinoid: No, because last time I tried I just got segfaults and oddness. 14:06
Infinoid Ah. Thanks, I was wondering whether these segfaults and oddness I'm seeing were due to my having used it wrong
jonathan Infinoid: I owe allison an attempted implementation of async for Rakudo, so she can see the issues we're running in to. Just didn't get to it yet. 14:07
Infinoid I'm writing a test for pipes, but I get an assertion failure when creating a thread. It can't find Test;Builder in some class hash 14:08
14:08 pjcj joined
Infinoid seems if I only clone the code and not all the other clone bits (see runtime/parrot/include/cloneflags.pasm) it gets farther, but that's a yak I don't need to shave right now 14:13
14:13 barney joined
dalek rrot: r39269 | bacek++ | trunk (3 files):
[pmc][core] Add function Parrot_pmc_try_reuse.

used in other PMCs to reduce GC pressure.
14:18
rrot: r39270 | bacek++ | trunk (2 files):
[core] dest can be null in Parrot_pmc_try_reuse.
14:22
bacek oh shi... 14:29
Do we have ticket about non-fully VTABLE copy to derived class? 14:30
14:31 autark joined
dalek rrot: r39271 | bacek++ | trunk/src/pmc.c:
[core] Preserve semantic of non-core PMCs in Parrot_pmc_try_reuse.
14:48
bacek ETOOMANYTYPOS...
pmichaud I'm getting a huge number of failures in trunk. 14:50
nopaste "pmichaud" at 72.181.176.220 pasted "test failures in trunk" (19 lines) at nopaste.snit.ch/16728 14:51
bacek pmichaud: r39270? 14:52
pmichaud yes.
trying 39271 now.
bacek it should fix it :/
(at least it passed on my box...) 14:53
15:00 whoppix joined
pmichaud yes, 39271 is much better, thanks. 15:01
bacek sorry for this... 15:03
anyway, rakudo's make spectest should be slightly faster at 39271 15:05
Coke needs to unleash bacek on partcl. =) 15:07
bacek Coke: one problem - I don't know tcl at all... 15:09
dalek rrot: r39272 | pmichaud++ | trunk/compilers/pct/src/PCT/HLLCompiler.pir:
[pct]: Update transcode option to HLLCompiler

  * Allow changing encoding as well as charset
15:41
rrot: r39273 | bacek++ | trunk/src/pmc/scalar.pmc:
[pmc] Reuse C<dest> when possible in Scalar.
15:54
rrot: r39274 | bacek++ | trunk/src/pmc/bigint.pmc:
[pmc] Reuse dest if possible in BigInt ops.
rrot: r39275 | bacek++ | trunk/config/gen/makefiles/root.in:
Run nqp_test after testing Parrot's core itself.
pmichaud bacek: does the nqp tests run only if all of the core tests pass? 15:55
bacek pmichaud: yes
pmichaud excellent bacek++
bacek I prefer to move all compiler's test (except imcc) behind core tests. But not today. 15:56
dalek rrot: r39276 | NotFound++ | trunk/examples/nci/xlibtest.pir:
[examples] add save as json feature to xlibtest.pir
15:58
bacek pmichaud: my latest commit shaved another 1.5-2 seconds from rakudo's "make test". I'm too tired to run spectest to check impact... But it should be slightly faster. 15:59
pmichaud bacek: excellent, that's a good improvement. I'm working on a fix that already shaves 2 minutes off of 'make spectest' 16:00
(actually shaves 2 minutes off of just one test file :-)
bacek pmichaud: technically, this 2 minutes came from "revamped tt452_reduce_multi" branch. 16:03
pmichaud bacek: no, I'm sure it didn't.
because I'm comparing times between my local changes using the same parrot in each instance.
i.e., I'm not comparing an older version of parrot with a newer one. 16:04
bacek pmichaud: it is :) I just start implementing "switch-base VTABLE for MULTI" in Pmc2c (instead of doing it manually as in branch) 16:05
pmichaud bacek: you're not reading what I'm writing. But no matter. You're wrong.
:-)
bacek But I'm fast! :) 16:06
pmichaud bacek: I start with a parrot. I time the test. I make a change *in Rakudo*. The test runs 2 minutes faster.
That tells me that the 2 minutes improvement didn't come from Parrot.
bacek hang on. Can you check rakudo before r39253 and at r39257? 16:08
pmichaud not easily. 16:09
I'll be more specific.
I run the test with parrot 39271.
I time the test (2m28)
I make a change to Rakudo
I time the test again (30 sec)
both tests are using parrot 39271 16:10
bacek Ah, Ok. It's different :)
pmichaud ergo, the 2 minute improvement is not due to the pmc2c changes
I'm not saying the pmc2c changes didn't produce an improvement, I'm simply saying that's not what I was measuring.
or that pmc2c changes aren't responsible for the 2m improvement that I'm seing. 16:11
bacek I'm talking about irclog.perlgeek.de/parrot/2009-05-30#i_1192684 (pmichaud
fwiw, Rakudo seems to have gotten faster since r39239)
:)
pmichaud sure, it did get faster.
And yes, it got about 2m faster.
but that was a different 2 minutes. 16:12
And that was 2 minutes for the entire suite, not for just one test file.
dalek TT #723 created by ingmar++: [PATCH] Parrot doesn't find ICU 4.2
bacek yes, this is performance boost that I've got from tt452 branch with manual converting MULTI to VTABLE-switch. 16:13
pmichaud which I find somewaht amusing, since allison did a lot of work last year trying to get the vtable-switches to be multis.
bacek So I just assumed that it was same things (just because they are same) 16:14
pmichaud (and yes, things slowed down significantly when that was done.)
16:15 Random joined
pmichaud still, I'm very happy for the speed improvements/recovery. 16:15
bacek "At your service" :) 16:16
Random ah, just came in the right second... I wanted to ask about the speed of loops in rakudo/parrot
following up on a post un use.perl.org I tested a simple while loop just incrementing an interger
in perl5 thats instant 16:17
in perl6 it's a whole-lot-of-memory and time
the code is as simple as: while ( $i < 1000000 ) { $i++; }
bacek Random: try very latest rakudo on top of very latest perl. It should be fast now
pmichaud "faster", maybe, but not "fast" 16:18
Random i just did a clone of rakudo 30min ago
and compiled with --gen-parrot
is that too old already?
pmichaud Random: we don't do much in the way of optimization.
(yet) 16:19
whereas perl5 is heavily optimized.
Random it can't be that the perl6 world is spinning that fast ;-)
pmichaud So it's not entirely an apples-to-apples comparison.
bacek Random: 30 minutes? It's like 18th century!
pmichaud bacek: keep in mind that Rakudo is still using a much older parrot
Random not optimzed is ok, but 1.4G of RAM and still running after 60seconds... is a little too extreme for me
ah ok, so I should also checkout parrot from svn and compile 16:20
and use that with my 30min old rakudo, right?
pmichaud well, "much older" in this case means about 12 hours
Random: even with the more recent parrot it's still likely to take a while.
There's just a *lot* going on behind the scenes that we haven't tried to work out yet. 16:21
bacek hm... It actually consumes too much memory
dalek rrot: r39277 | Infinoid++ | branches/io_rewiring/src/pmc (2 files):
[io] Clean up Pipe and PipeHandle some more.
rrot: r39278 | Infinoid++ | branches/io_rewiring/t/pmc/pipe.t:
[io] Add a test for the Pipe and PipeHandle PMCs. It only checks object creation and accessor methods for now.
rrot: r39279 | Infinoid++ | branches/io_rewiring/PBC_COMPAT:
[io] Bump PBC_COMPAT for the new PMCs we've added to this branch.
rrot: r39280 | Infinoid++ | branches/io_rewiring/t/native_pbc (4 files):
Regenerate PBC files after to PBC_COMPAT bump.
pmichaud although I am surprised about using 1.4G of RAM, that sounds like a serious memory leak
Random well it definitly is 16:22
Infinoid I see that much usage here, too
Random I'll retest with the latest parrot now
pmichaud but the biggest reason for the loop being slow is that it's resulting in several million Parrot subroutine calls
Infinoid I suspect it might be more extreme on x86-64 than on x86-32
pmichaud and Parrot subroutine calls are known to be especially slow.
(those several million Parrot subroutine calls in turn are translating to several-several-several-million C function calls)
Random I'm running on amd64 16:23
still in perl5 it's optimized, so it's probably 1mill perl incr for 1mill c calls 16:24
bacek pmichaud: (and "< 1000000" creates new Integer on every loop)
Random but even if parrot is not optimized yet and has calling overhead that would be too extreme
bacek wishes for SSA and CSE for PBC... 16:25
Random the code actually was: my $i = 0; while ( $i < 1000000) { $i++ }
so the $i should be instanced once
or is there something wrong with the comparison on a constant? 16:26
Infinoid wishes his brain had hardware acceleration for decoding TLAs
bacek afk # .zzz
Infinoid sleep well bacek
pmichaud $i < 1000000 becomes a call to 'infix:<'($i, 1000000)
so that's 1 million subroutine calls right there.
Random ok, and these are extremly slow and leak mem 16:27
pmichaud yes.
then $i++ becomes 'postfix:++'($i)
bacek pmichaud: not quite true...
loop32_test:
find_lex $P23, "$i"
new $P24, "Int"
assign $P24, 1000000
$P22 = "infix:<"($P23, $P24)
and Parrot bails out... 16:28
Random bacek: how did you generate that?
pmichaud bacek: the fact that the 1000000 becomes a PMC is in fact irrelevant to the timing
(unless we decide to do a loop optimization to factor it out of the loop) 16:29
bacek pmichaud: it relevant. GC isn't bloody fast.
pmichaud bacek: I'm saying that even if we generated
Random When is optmization planned for the basic loops and functioncalls?
pmichaud $P22 = "infix:<"($P23, 1000000)
we'd _still_ end up converting that 1000000 to a PMC on each call.
(it would just be done by the calling conventions instead of explicitly in PIR) 16:30
Random *puh*, I guess svn checkout is faster then svn up of a month old parrot ;-)
pmichaud bacek: so the fact that the 1000000 becomes a PMC is true in both cases.
bacek pmichaud: we should not. 16:31
pmichaud bacek: depends on how you define "should not"
bacek Proper way is optimise it on PIR/PBC level
pmichaud if you're saying that we should factor the constant out of the loop, I likely agree. But that's not code that is particularly important at the moment.
Random is there a way to print how many PMCs were generated when terminating the program?
pmichaud Random: chromatic has a way of doing it -- I'm not sure what he does. I think there might be an option to Parrot to do it, though. 16:32
You can always run perl6 as parrot/parrot perl6.pbc instead of "./perl6"
and then parrot options can be placed before the "perl6.pbc"
bacek felt asleep
pmichaud bacek: I'm fine with optimizing it on a PIR/PBC level. But we don't do that yet.
Random ah ok
Infinoid Parrot's -D option should report that, but it doesn't seem to work at the moment (parrot->pdb is NULL) 16:34
Random trying to compile the latest parrot r39280 fails for me 16:45
In file included from src/string/api.c:29: 16:46
src/string/private_cstring.h:31:6: error: invalid digit "9" in octal constant
31: {0759094, "__get_string"} 16:49
guess that should not be a octal, although it starts with a zero
correcting it by removing the leading zero in the file fixes that, but the header of the file says it's autogenerated, so I guess that should be fixed somehwere else 16:50
Infinoid yeah, it's generated by c2str.pl 16:51
Random and now ./miniparrot gives me a segmenation fault :-(
6293 Segmentation fault ./miniparrot config_lib.pasm > runtime/parrot/include/config.fpmc
guess miniparrot did not like my corrections 16:52
Infinoid I'm not surprised. That first number is supposed to be the string's *length*.
So your value looks a little bit large
Random hehe, well then I guess I should correct it to the length 16:53
Infinoid that fix won't last, I'd rather find out why it was wrong
Infinoid tries a trunk build
Random yes, just saw that *alot* of lengths are wrong for the strings 16:54
{631, "get_number_keyed_str"},
should be that way either probably
Infinoid sounds pretty broken 16:55
You mentioned being on amd64, I think? Linux?
Random yepp 16:56
Infinoid same here. (gentoo)
Random Linux einstein 2.6.26-1-amd64 #1 SMP Sat Jan 10 17:57:00 UTC 2009 x86_64 GNU/Linux
It's debian testing + a little unstable
Infinoid I just did a fresh trunk build and it worked fine
Random c2str,pl did then something odd, right? 16:57
Infinoid Can you try "make realclean", reconfigure, rebuild and all of that, to see if the problem persists?
Random of course
I used that working copy beofre for a build... so maybe... make realclean && perl Configure.pl --prefix /opt/parrot && make 16:58
let's see it helps
*if it helps
Infinoid Oh, you mentioned having updated from last month's parrot, right?
Did you do a realclean/reconfigure as part of that process?
I made some changes to c2str on Apr 25 which modified its file formats and that would have required a realclean. (But realclean is a good policy every time you update, regardless) 16:59
c2str used to generate a "hash" value that wasn't used by anything, and it may have been parsing that from the .str files thinking it was the length. That got removed about a month ago, but the old .str files would have confused the new c2str 17:01
Random yepp, it IS as good policy 17:03
forgot that today
so it's built
Infinoid no miniparrot segfault?
Random do I need to rebuild rakudo with the newer parrot now? 17:04
nope, everything fine now, you did a good job ;-) 17:05
Infinoid you should reconfigure and rebuild rakudo when you rebuilt the parrot it depends on, yes
Random is it enough if parrot is in the path?
Infinoid there's a --parrot-config option to tell rakudo where to find parrot 17:06
Random and where is that parrot-config file? 17:07
i did a make install to /opt/parrot
ah typo, found it 17:08
let's see to improvements for my simple benchmarks ;-)
dalek rrot: r39281 | Infinoid++ | trunk/config/auto/icu.pm:
Apply patch from ingmar++ in TT #723:

ICU 4.2's 'icu-config --prefix' outputs an extra newline, strip it. Otherwise this breaks as follows: icuheaders: captured /usr Unsuccessful stat on filename containing newline at config/auto/icu.pm line 332. Unsuccessful stat on filename containing newline at config/auto/icu.pm line 337. For icuheaders, found /usr
  /include and 1
17:10
TT #723 closed by Infinoid++: [PATCH] Parrot doesn't find ICU 4.2 17:12
Infinoid feather.perl6.nl/~azawawi/padre-perl6-exe.png is shiny! 17:15
Random well I guess, the 100000 incr running in a loop are still very unperformant... 57sec and roughly 1.3GB of RAM for 100000 increments of variable. 17:20
time ./perl6 -e 'my Int $i = 0; while ( $i < 100000 ) { $i++; }'
Infinoid Patches welcome. :)
Random so still the same with latest rakudo and parrot, is there some kind of continious benchmark run with the smoke tests? 17:21
to see when performance is increased (or decreased)
hehe, guess that is a little too big a job for me
Infinoid I'm not sure. There is a t/benchmark/benchmarks.t but I don't think it's run as part of normal "make test", and I'm also not sure it outputs any stats in its TAP output 17:22
Random guess it should be a seperate run like make benchtest or something 17:24
as it's output would need to be gathered and compared to previous versions 17:25
nopaste "infinoid" at 24.182.55.77 pasted "t/pmc/packfileannotations.t failures on linux/amd64" (27 lines) at nopaste.snit.ch/16729
"infinoid" at 24.182.55.77 pasted "t/pmc/packfileannotations.t failures on linux/amd64 (with stderr too, this time)" (23 lines) at nopaste.snit.ch/16730 17:26
Infinoid wait, what? intermittant failure...
nopaste "Infinoid" at 24.182.55.77 pasted "Try again..." (29 lines) at nopaste.snit.ch/16731 17:27
Infinoid msg bacek Any idea why I'd be getting an intermittant failure here? nopaste.snit.ch/16731 (your branch merge is the only recent thing I could see in the logs) 17:28
purl Message for bacek stored.
Random does anyone know which runcores are currently supported for parrot? 17:29
fast is the default I guess, and jit needs a jit-enabled core
most of the ones specified in the parrot -h output are not available 17:30
Infinoid it varies depending on your platform.
muixirt Random, there is tools/benchmark.pl
Infinoid trace, slow, fast are typical, cg and cgp are available on (most/all) gcc architectures, jit is x86-32 only by default (I think) 17:31
Random when switching runcores, parrot has to be run for the build directory, otherwise it says it cant find PCT.pbc 17:33
muixirt how is the status of jit - it worked better in the past, if i remember correctly
Random well cgp is 1 second faster... as fast 17:34
but at least there is room for optimization
let's see about that when chromatics changes land
Infinoid jit is very difficult to keep maintained, at this point most devs are leaning toward replacing it with llvm or libjit, or reimplementing it on top of a very small (RISC-like) set of instructions 17:35
Random that would definitly be a good idea
muixirt Infinoid, it jitted only one opcode, right?
Random as those other implementation get also attention outside of the parrot world
Infinoid no, it jits a few. But still a rather small subset of the whole 17:36
17:36 Random left
Infinoid the fact that we have more than a thousand ops doesn't help 17:37
muixirt but another vm (llvm) under the hood of the parrot vm sounds ... well ... 17:39
Infinoid We'd be inclined to prefer a solution that gives us the easiest jit access with little to no extra baggage
muixirt i like the v8 javascript vm solution: compiler -> machine code :-) 17:41
Infinoid "libJIT has some advantages (just a JIT, arguably better C bindings, more standalone, perhaps less JIT startup disadvantages (no clang to get some JIT for free, fewer developers)." -- Chromatic, groups.google.com/group/parrot-dev/...ba5a096172 17:44
oops, I somehow missed a "time, closer to our existing JIT but more cross-platform) and some " in the middle of that quote
Coke bacek: (not know tcl) it's mostly PIR. You can just run one of the examples and clean up the resulting parrot mess. =) 17:45
(printing # of pmcs) I have a macro for that. 17:48
code.google.com/p/partcl/source/bro...os.pir#103 17:49
muixirt Infinoid, the problem is that you lose information with every layer between source code (Perl6) and the CPU, and that might be bad for optimization 17:52
Infinoid Right. Hence jit.
muixirt Infinoid, erm i meant the chain perl6 -> pir -> pbc -> parrot -> llvm/libjit won't give you much speed 17:55
Infinoid Ok. We do have very good annotations support which I'm hoping optimization can make use of, but we do very little optimization in general 17:56
(at the moment)
Infinoid is running callgrind on Random's perl6 while-loop 17:57
muixirt but this data has to be (at least partl) processed in each layer, that consumes memory and cpu cycles 17:59
*partly
Infinoid That's why we want to separate as much of the overhead as possible from runtime, and do it at compile tile 18:01
time
typically the runtime chain will be: pbc -> parrot -> jit 18:02
In that simple while/increment loop, we do indeed spend a lot of time creating and destroying things. 18:15
13.19% spent in pmc_new (and functions it calls) 18:16
10.51% in free_pmc_in_pool
9.83% in Parrot_Hash_init
the next biggest thing after object creation/destruction is hash access; 8.64% in parrot_hash_get_bucket 18:17
but since we're calling it 6.2 million times (for a loop of 10000), that's probably not its fault. 18:19
muixirt cool 18:20
i imagine that this loop would run on my first pc (486sx-33) ... it would be better counting for myself ;-) 18:22
muixirt praises Intel for the speedups of their cpus 18:23
Coke hurm. make -j 2 install-dev fails. 18:27
18:28 iblechbot joined
muixirt Infinoid, but the problem with the loop isn't parrots fault, i think 18:28
Infinoid No, but if we can make parrot faster for this workload, a lot of other similar workloads will benefit 18:29
Coke tries to eliminate TclObject 18:30
bah, failure. 18:31
muixirt Infinoid, true, but can you?
Infinoid At the very least, I can make a few tweaks and see what happens... if it improves things by more than a couple of percent, I'll check it in 18:32
(but I doubt it, as chromatic and others have already been over this stuff) 18:34
muixirt i believe making any random combinations of opcodes (maybe produced by a rakudo compiler gone wild) run fast is *very* hard
Infinoid Right now I'm just looking at saving a few cycles in parrot_hash_get_bucket 18:35
Coke that's tcl's #1 c function, as I recall.
pmichaud actually, Rakudo doesn't produce a lot of opcodes. 18:36
that's true for most compilers in general, fwiw -- compilers tend to use very small numbers of opcodes.
it's easy to check. 18:37
nopaste.snit.ch/16732 18:41
the ones that begin with a '$' are in fact function calls
oops, and some of them are calls to root_new 18:42
muixirt looks at the generated pir code 18:44
looking at mips asm code would be easier ;-) 18:46
nopaste "pmichaud" at 72.181.176.220 pasted "generated pir for loop" (136 lines) at nopaste.snit.ch/16733 18:47
pmichaud _block14 has the main loop 18:48
18:48 ewilhelm joined
pmichaud the section called loop32_test is the loop itself 18:48
you can see that's pretty tight already. 18:49
muixirt sub "_block25" :anon :subid("12_1243709245") looks suspicous, it does the incement?
pmichaud yes, it does the increment
muixirt and allocating these registers is so costly? 18:52
pmichaud the costly part is the parrot subroutine calls
for this loop, we have the following calls 18:53
"infix:<"
$P26() ( the call to _block25 )
"postfix:++"
those are expensive. 18:54
at least, that's the only thing that I see here that can explain why the loop takes so long.
18:55 tetragon joined
pmichaud that tells me that it's parrot that is slow, not Rakudo. I'm hard-pressed to see how Rakudo could generate much tighter code (in the general case) than what it's generating now. 18:55
Coke the PCC are expensive.
muixirt but why the $P26() ? it seems so superfluous 18:57
pmichaud you mean, why make the call to another parrot sub?
inlining it would end up being an optimization, yes. But that optimization is worthless if we have instead
muixirt yes, just for incrementing $i
pmichaud my $i = 0; while ($i < 1000) { my $j; $i++; }
because we need another block in which to create the $j
Coke it would be interesting to time it inlined and compare. 18:58
see if figuring out how to do that optimization now is worth it.
muixirt $j ?
pmichaud the body of the while loop has its own local $j lexical
parrot lexical scopes have to be separate subs
Infinoid I suppose you could cache the .lex lookup for $i, and you could cache the 1000 constant pmc 18:59
Coke muixirt: the current sample doesn't have $j - this is just an example of why we need it in the general case.
Infinoid But the loop is indeed pretty tight overall
Coke need it; it == the sub.
pmichaud Infinoid: I can cache the .lex lookup for $i *as long as* the inner loop doesn't rebind it to a different PMC
18:59 Whiteknight joined
Infinoid If the inner loop is a different scope, does the outer loop inherit that change? 18:59
(I guess I don't fully understand binding.) 19:00
pmichaud Infinoid: my $i = 0; while ($i < 1000) { $i := 1000; }
Infinoid uh, /me =~ s/loop/scope/
pmichaud that causes the symbol $i to point to an entirely different PMC from the one that held the zero
so if the while loop has a register that continues to point to the zero, we get an infinite loop. 19:01
Infinoid oh, ok. How expensive is the .lex lookup, anyway?
pmichaud it shouldn't be that expensive.
jonathan It's a lot pricier than it need be.
Infinoid I won't worry about it then 19:02
pmichaud it's pricier than it ought to be, yes, but it's not huge.
but I can do some benchmarks, just a sec.
Infinoid if you had to search a big stack of lexpads, or whatever, you could always have a revision number that got bumped every time you do a bind, and invalidate your cache if the rev was increased
jonathan Given that we statically know to look down the outer chain one and then statically know which register number to look in.
pmichaud Infinoid: afaik, there's not a way to know that something is rebound
Infinoid But I'm not a big fan of making things more complicated like that, generally
jonathan But instead, we do two hash indexes.
Which (surprise!) costs more than a pointer deref and an array index. 19:03
pmichaud and then instead of doing find_lex, we'd always be checking the cache, which is probably the same thing.
Infinoid Rebinding calls LexPad.set_pmc_keyed_str(), I'm guessing?
That's all we need to detect
pmichaud Infinoid: and in the outer loop...? 19:04
jonathan I don't know that we need to cache it, so much as use the information we statically have available to not have to do lexical lookups.
As in, to not have to do them by name every time.
pmichaud jonathan: you're talking about something different than we are.
Infinoid is talking about moving the find_lex $P23, "$i" outside of the loop
jonathan Oh. How would that work? 19:05
pmichaud that's my point, it wouldn't.
jonathan Ah.
:-)
nopaste "pmichaud" at 72.181.176.220 pasted "factor out find_lex" (13 lines) at nopaste.snit.ch/16734
muixirt stubbornly persists, rakudo should have optimized that loop away ;-)
pmichaud Infinoid: the problem is, if something rebinds the '$i' lexical, how do we reflect that change in $P23 ? 19:06
jonathan muixirt: Well, if you want to start hacking on a Rakudo optimizer... ;-)
pmichaud i.e., how do we know that the PMC in $P23 is no longer the one for $i ?
Infinoid Right, you need a (somehow faster than .lex) way of detecting that case
pmichaud if we put in additional PIR instructions to detect that case, then we're probably better of having just kept the 1-instruction "find_lex" inside the loop. 19:07
jonathan *nod*
pmichaud *better off
Infinoid So my (completely ugly and hackish) thought was, you keep a handle on the current PMC for $i, as well as the LexPad that holds it and the old revision of that LexPad... then you bump the LexPad revision whenever you overwrite anything in that lexpad. So you check LexPad.get_revision() or whatever against your previous version
pmichaud Infinoid: but *where* do we do that check?!? 19:08
Infinoid you do that check once per loop
pmichaud then why not just do the find_lex ?
Infinoid The goal is to make that check less expensive than just doing .lex directly
or find_lex, sorry
pmichaud find_lex can be made much faster
Infinoid Yeah, that would be better for all parties involved :)
pmichaud and find_lex could potentially be the thing that does the check 19:09
either way, that's a *parrot* issue and not a Rakudo code-gen one.
jonathan Ideally, we'd only call $P0 = find_lex '$name' occasionally.
pmichaud ideally, once per thread-of-execution
jonathan And an optimizer during the PIR -> PBC phase would re-write a lot of them as $P0 = find_lex 1, 2 or similar.
Infinoid Is it possible for rakudo to detect whether any binding will occur in the inner block, and simplify the generated loop? 19:10
pmichaud Infinoid: no.
jonathan "1 outer down, register number 2"
pmichaud because rebinding could also occur in any function that the inner block happens to call.
jonathan pmichaud: Only if it were marked "is context" though, no?
Infinoid Right, you could only detect it in the case that the inner block doesn't *call* any functions
pmichaud Infinoid: and doesn't call any operators, either. 19:11
jonathan: no, even without "is context"
jonathan Infinoid: The problem is that $x++ *is* a call.
pmichaud: From Perl 6?
pmichaud: How?
pmichaud my $i = 0; sub foo() { $i := 1000; }; while ($i < 1000) { foo(); }
jonathan oh
Yes, that's harder
However, also lexically known. 19:12
pmichaud oh?
Infinoid So a potential optimizer would have to inspect the contents of any functions called, and detect the case where any functions would be overwritten at runtime, in order to make this determination
pmichaud my $i = 0; multi sub foo(...) { $i := 1000; }; while ($i < 1000) { foo(); }
it's not just functions called, but also all possible functions they could call.
Infinoid yeah
pmichaud which can dynamically change
jonathan pmichaud: Yes, true.
pmichaud because we have virtual methods
so we _can't_ know at compile time. 19:13
jonathan Aye.
pmichaud (except if we can somehow determine that every operation involved is static and hasn't been monkeypatched and ....)
jonathan So basically Perl 6 is unoptimizable.
Coke kind of like tcl! =-)
pmichaud oh, there are optimizations to be had, yes -- but avoiding lexical fetches isn't one of them in our current model (that is somewhat imposed upon us by Parrot) 19:14
Infinoid beautiful. I'm looking forward to seeing what kinds of optimizers we can build on parrot
jonathan pmichaud: OK. But that doesn't change my suggested approach to lexical lookups that we statically known the position of.
pmichaud jonathan: agreed.
The answer here is to make find_lex fast.
jonathan (A Parrot-level op rather than a Rakudo-level one though.)
pmichaud or to give Parrot a smarter overall lexical system 19:15
however, let's do a few benchmarks....
argggh, I can't run the PIR code anymore from the command line. :-( 19:16
jonathan Going back to the while loop example that kicked all of this off though - it'd be *really* helpful to know where/how that leaks so much memory.
pmichaud oh, I can force a loadlib
vi nopastes coming 19:17
nopaste "pmichaud" at 72.181.176.220 pasted "basic benchmark, no changes" (145 lines) at nopaste.snit.ch/16735 19:18
"pmichaud" at 72.181.176.220 pasted "avoiding the call to the nested block" (138 lines) at nopaste.snit.ch/16736 19:20
pmichaud actually, that's a little over-optimistic 19:21
nopaste "pmichaud" at 72.181.176.220 pasted "avoiding the call to the nested block, corrected" (140 lines) at nopaste.snit.ch/16737 19:22
pmichaud so, the 10,000 calls to the nested block cost us .140s
factoring out the constant.... 19:23
nopaste "pmichaud" at 72.181.176.220 pasted "factoring the 10000 constant out of the loop (6.903s)" (140 lines) at nopaste.snit.ch/16738 19:25
pmichaud purl: 7.011-6.903
purl 0.108
pmichaud creating the 10000 Ints cost us .108 19:26
hmmmm, I'm curious.
muixirt so whats left (i cant really follow) 19:27
pmichaud I don't know, I'm a bit surprised myself.
maybe it's the postfix:++ that is expensive?
it would be really nice if we had a pir-level profiler for this st..... oh, right. :-P 19:28
Infinoid tries adding some branch prediction annotations to parrot 19:30
19:30 jan joined
pmichaud to do one loop (instead of 10000) takes 0.728s 19:30
I wonder if the postfix:++ is the costly part, as it has to ultimately go through the vtable interface to work 19:31
muixirt i once coded such a loop in asm for a direct threaded code model and it took 46 secs! 19:32
pmichaud a-ha! 19:33
muixirt well it counted to 1000000000 :-)
19:34 Zak joined
nopaste "pmichaud" at 72.181.176.220 pasted "original PIR, replacing "postfix:++"($P30) with inc $P30 (jonathan++ notice!)" (142 lines) at nopaste.snit.ch/16739 19:34
pmichaud so, it's the call to postfix:++ that is super-expensive. 19:35
...and looking at the postfix:++ call, I think it's no longer accurate. 19:36
I thought I had fixed this already. :-(
jonathan pmichaud: So adding this: 19:37
.loadlib 'perl6_group'
.loadlib 'perl6_ops'
Was what helped?
pmichaud to run the PIR directly from parrot, yes.
not for the speed improvement.
the speed improvement comes from
jonathan pmichaud: I'm a little surprised we need them at this point, but OK.
pmichaud jonathan: we shouldn't need them -- it's a known Parrot bug. 19:38
TT #150 19:39
jonathan ah, ok
pmichaud the speed improvement comes from
$ diff x.pir x4.pir 19:40
104c104
< $P31 = "postfix:++"($P30)
---
> inc $P30
$
...which makes me suspicious of vtables and multidispatch
jonathan pmichaud: Right, but in the general case that isn't going to fly?
pmichaud right, we really shouldn't generate "inc" here
my point is imply that something about "postfix:++" makes it expensive.
*simply 19:41
jonathan Oh, we are generating inc and not "postfix:++"( )?
pmichaud no, we generate "postfix:++"()
jonathan OK, that's right...but yes, I can appreciate that may be costly.
pmichaud when we do that, my benchmark takes 7.15 seconds
jonathan That's because postfix:++ does...
1) Call .succ
pmichaud if I replace the "postfix:++"() with "inc", the benchmark takes 1.x seconds
jonathan 2) Do an assignment 19:42
IIRC.
pmichaud it also makes a clone
(to return the original value to the caller)
jonathan right, so it can...right.
pmichaud however, for some reason we're cloning it twice. 19:43
jonathan pmichaud: copy op in assignment.
pmichaud no, I mean in postfix:++ itself.
there are two calls to clone
jonathan so I see 19:44
pmichaud I should revisit this.
jonathan Aye, there's almost certainly a way to make that more efficient.
pmichaud but something in there is responsible for 80% of the execution time
19:44 Austin_Hastings left
pmichaud and we could probably optimize the postfix:<++>(Int) case 19:45
jonathan *nod* 19:47
pmichaud: We gotta be a tad careful of this case:
my Even $x = 4; $x++; # should die 19:48
(provided you had defiend your subset Even)
pmichaud oh, I'll still do the assignment.
jonathan Sure.
pmichaud but instead of clone + call inc, I'll probably just add one and assign that. 19:49
jonathan *nod*
pmichaud that would *also* help with our BigInt boundary conditions, I suspect :-)
well, I need a break for a while. I'll come back to this later today or tomorrow. 19:50
also need to figure out how to get the ucs2 encoding to work. There are a lot of the spectests that have non-breaking spaces in the comments, and thus run slow. 19:51
s/run slow/parse slow/
jonathan OK, have a good break. 19:52
I need to track down a segfault next, so will probably break for the day too...
jonathan looks at something and boggles 20:05
In Parrot's src/call/pcc.c
set_retval(PARROT_INTERP, int sig_ret, ARGIN(Parrot_Context *ctx))
Inside here do do this:
call_state st;
We pass a pointer to it...
if (set_retval_util(interp, "P", ctx, &st)) {
set_retval_util does 20:06
Parrot_init_arg_op(interp, ctx, src_pc, &st->src);
...hang on, my head is exploding... 20:07
So at this point we've passed what woulda I guess been st.src in the original - well, rather, a pointer to it.
hmm...it may be OK. Ignore me. :-S 20:08
pmichaud ("did someone say something...?") 20:21
dalek TT #724 created by pmichaud++: [bug] Parrot fails numeric conversion of ucs2 strings 20:51
21:10 cognominal joined
Infinoid oh, wow. 22:10
I sped up that rakudo loop by 1 second by putting (C-level, gcc-specific) branch prediction annotations in a few key places, and then sped it up another 4 by playing with the Hash internals allocation a bit
I need to re-analyze the branch prediction stuff. The profile I had based it on was without any kind of gcc optimization at all, and it looks like the code generated in --optimize mode is quite a bit different 22:12
pmichaud Infinoid++
cotto It'd be pretty cool to get a pgo build using the data from the test suite.
Infinoid Either way, expect a patch from me in the near future to introduce (linux kernel-like) likely() and unlikely() macros
cotto I was thinking the same thing.
Infinoid (which do nothing on non-gcc, for now)
I'm going to add both of these patches as tickets, rather than applying them directly, as neither of them are particularly pretty 22:14
cotto That's what you get for imitating kernel code. 22:28
dalek TT #725 created by Infinoid++: [PATCH] Add c-level branch prediction annotations 22:30
22:31 rg1 joined
Infinoid The kernel does it pretty well. I just need to redo the callgrind profile to find useful places to use it 22:37
it only saves about half a second as-is
(most of that is probably in PARROT_ASSERT and ASSERT_ARGS)
#726 is the one that could use some more eyeballs (sped up that rakudo benchmark by around 10%) 22:41
dalek TT #726 created by Infinoid++: [PATCH] Reduce calls to mem_sys_allocate() when creating a Hash 22:44
jonathan Infinoid: More notably, hashes are used heavily all over the place.
Infinoid Yeah, that's why I focused on it
jonathan Infinoid: So making hashes faster likely has a real impact on a lot of code.
Infinoid++ 22:45
Infinoid Note: I haven't even looked at why the rakudo benchmark uses up so much memory yet, that's likely going to make a bigger impact
jonathan Yeah, agree.
I'd turn on context debugging and see if we're leaking a bunch of those.
Infinoid Oh, we still don't have context PMCs, do we? 22:46
jonathan No
Infinoid Ouch.
jonathan But my other concern is...
I think somebody pointed out huge GC overhead.
If we're somehow leaking contexts but leaving the PMCs they pointed to reachable somehow, that could give huge GC pressure.
Infinoid hmm. If that were true, I think I'd see more GC activity in callgrind 22:47
One sec, rerunning callgrind
jonathan Ah, I thought there was a lot?
Infinoid my previous callgrind invocations were for a non-optimized parrot, but there didn't seem to be a ton 22:48
allocations and frees were more or less proportional
jonathan Ah, OK. 22:49
Infinoid jonathan: irclog.perlgeek.de/parrot/2009-05-30#i_1194677
sadly, I can't get text output from kcachegrind. I can post a screenshot when this one finishes tho
22:49 Austin_Hastings joined
jonathan 13.19% spent in pmc_new (and functions it calls) # this is a lot of time, given it's essentially just initializing stuff rather than doing useful work... 22:51
If you add it up, we spend a quarter of our execution time managing creating/freeing PMCs. 22:52
Infinoid yes, but then again, it is called over 6 million times
jonathan 6000000 / 10000 22:53
purl 600
jonathan Average 600 PMCs per loop?
OK, we'd need to subtract what we cost just starting up etc...
Infinoid yeah, I can work on that next 22:55
I got the stats slightly wrong... 2.4M calls to pmc_new(), 6M calls to parrot_hash_get_bucket()
Please see quack.glines.org/upload/infinoid/c...-10000.png
jonathan wow, pretty! 22:56
Infinoid parrot_create_hash shrank a bit from the last time, but it's still a fairly big block on that chart
jonathan Hmm. It spends a lot of its time inside malloc. 22:57
Infinoid parrot_gc_sweep_pool weighs in at around 3%
jonathan Parrot_Class_isa?
That seems to have some percent too 22:58
I can't quite tell
Can only se Parrot_Class_i on the diagram...
Infinoid about 2% in Parrot_Class_isa and Parrot_Class_isa_pmc in total
there's a Parrot_Class_isa_pmc and a Parrot_Class_isa_pmc'2, I dunno what the difference is (I'm pretty new to this tool) 22:59
I think it's the same thing but from a different call chain
jonathan ok
Parrot_new_str_COW is also the other one that looks sizable. 23:00
Infinoid overall, I think our overhead is pretty well spread out.
jonathan Yes, for the most part it seems so.
I guess chromatic++ has got a lot of the hot spots tracked already. 23:01
Infinoid yeah, and he's been advocating this tool for some time now
I can see why.
jonathan yeah, me too after that screenshot. 23:02
Infinoid Parrot_new_str_COW is sizable mostly because it's called more than 3 million times
which results in more than 2 million calls to Parrot_gc_new_string_header
oh, it calls it 3 million times, just from 2 different places 23:03
jonathan Oh?
Infinoid yeah, there's an if/else based on whether the passed in string was constant or not 23:04
jonathan ah, ok
Infinoid So, since Hashes are so incredibly high-profile, what do you think about the TODO on src/hash.c line 981 regarding optimizing for lots of small hashes? 23:05
jonathan would love to have output in this style at PIR level
Infinoid uh, that's line 981 in my build, but I have a couple of patches applied here
972 in trunk 23:06
jonathan yeah, I see th eone
Trying to work out exactly what it means 23:08
I think what it's driving at is trying to do 1 malloc not 2.
Infinoid ah, so it's pretty much what I did in TT #726 23:09
But I didn't stick it in the struct, I just tacked it onto the end of the buffer
jonathan No, I think it might be meaning "don't allocate a separate bucket store" 23:10
Infinoid Yeah, that's what I changed, but they're suggesting reducing the bucket store size as well 23:11
jonathan The comment below that leads me to wonder if Hash generally is doing stuff to make OrderedHash easier.
Infinoid hmmm.
Infinoid tweaks the number of buckets and redoes the benchmark
jonathan I'm no great data structures guy but part of me wonders... 23:13
HashBucket *bs; /* store of buckets */
HashBucket **bi; /* list of Bucket pointers */
From what I can see, we're manintaining an array of the buckets
And also maintaining a linked list.
Infinoid yes. After my patch, *bs is a pointer to the memory directly following the Hash struct itself 23:14
they're both allocated in one go
Coke I wonder again why we're not just using a hash library. =-)
jonathan Coke: Because concaine libraries are so much better!
Infinoid *bs ends up pointing somewhere else when we expand the hash, but initially, they're both in the same buffer
jonathan OK
Infinoid By the way, changing INITIAL_BUCKETS from 16 to 8 speeds us up by another 3 seconds (another 7% or so) 23:15
23:15 snarkyboojum joined
jonathan I think you're not the first person to observe this. 23:15
But not sure why the change didn't make it in before...
In reality most hashes seem to be quite small. 23:16
Coke isn't that going to depend on the benchmark you're running?
Infinoid eh, 2 seconds on average
Yes, absolutely
jonathan Coke: Yes, it is.
Infinoid: How icky is it to lazily allocate the bucket store? 23:18
That is, not allocate it until we actually store something?
nopaste "Infinoid" at 24.182.55.77 pasted "Ooh pretty numbers!" (18 lines) at nopaste.snit.ch/16741 23:19
jonathan I ask because a lot of the time, e.g. whenever there's a :named :slurpy, there's nothing put into it.
cotto I like laziness because it reflects the way I work. 23:21
Infinoid jonathan: That sounds a bit more involved. We don't really pay anything extra now that it's allocated in the same buffer as the Hash
If we were to introduce lazy allocation, I don't think the NULL checks would slow things down *that* much, but I don't think we'd gain anything either 23:22
dalek rtcl: r400 | coke++ | trunk/runtime/tcllib.pir:
the NS referenced here is long gone.
Infinoid It would help memory usage and hurt performance slightly.
performance is probably a wash 23:23
jonathan: Didja look at my pretty shiny numbers?
jonathan OK. It's just that it'd save us a malloc and the initialization work (though you already made that cheaper though).
And afaik PGE heavily uses the empty-slurpy-hash pattern. 23:24
Infinoid it won't save us a malloc any more, we'd just be allocating slightly less
jonathan Your shiny numbers are interesting.
Oh, OK. In that case, then probably not wroth it then.
*worth
I think it may be worth running some other comparative benchmarks. 23:25
Rather than just optimizing for this one.
But in Rakudo generally, small hashes is most likely the trend.
Infinoid Yeah, absolutely 23:26
jonathan (Not least because of the PGE has a lot of empty ones issue.)
(And Rakudo is about to get %_ on all methods...)
Infinoid what's %_? argument hash? 23:27
jonathan By S12, all Perl 6 methods are meant to get a default slurpy hash parameter. 23:28
(Which is normally not so interesting, but gets more interesting now we are about to have deference.) 23:29
jonathan loves how "Windows" doesn't even appear on the valgrind platforms to port to page. 23:30
Infinoid hah
valgrind emulates the hardware and the OS, and porting it is quite a lot of work, so they're pretty selective
jonathan Aye, understandable. 23:31
Infinoid I didn't see much difference between 16 buckets and 4, from examples/benchmarks/primes.pasm, after bumping its loop limit up to 20000. 4 buckets was slightly faster, but no more than 3% 23:32
Infinoid looks for more benchmarks 23:33
jonathan Sure. Performance regressions are probably the more interesting issue.
Infinoid Any suggestions on how to find those? 23:34
Lots of big hashes would probably be the losing performers here
fairly evenly distributed datasets, forcing lots of calls to expand_hash 23:35
jonathan *nod*
Not sure where I'd look for them. 23:36
Did you try benchmarking say Rakudo make test?
Infinoid I'll do that now 23:37
between trunk and trunk+TT725+TT726, rakudo "make test" got about 200ms faster. It got another 100ms faster from changing INITIAL_BUCKETS from 16 to 4. 23:46
I don't trust those numbers tho, they're less than the margin of error
Infinoid does an average of 10 runs each
jonathan not a regression but not a noticable win either. 23:47
Infinoid yeah, the effects were much more noticable in the while loop case 23:48
actually, based on my initial test, TT725 may have regressed it slightly and TT726 brought it back, I'll know in a few minutes
23:53 particle joined