github.com/moarvm/moarvm | IRC logs at irclog.perlgeek.de/moarvm/today
Set by moderator on 8 October 2013.
00:15 benabik joined, ggoebel6 joined 00:26 FROGGS joined 00:58 benabik joined
dalek arVM: d525bd2 | jimmy++ | src/math/bigintops.c:
Add missing MVMROOT to MVM_bigint_radix
 01:56
diakopter JimmyZ: that seems to fix the problem folks were having on the mac 02:00
JimmyZ: good find :)
JimmyZ: may I ask how you diagnosed and found it?
JimmyZ diakopter: I see jnthn++ fixed MVM_radix in the commit message, so I remember MVM_bigint_radix should be fixed too, since I ported these two to MoarVM 02:07
diakopter oh :) 02:08
JimmyZ ;)
diakopter oh. 02:10
nope, hit another one on the mac 02:11
oh wait
no, it's the same one
nm, sorry for the false hope :)
JimmyZ :P 02:12
03:17 foo_bar_baz joined 03:42 jnap joined 07:02 FROGGS[mobile] joined
FROGGS[mobile] o/ 07:07
09:33 FROGGS[mobile] joined 10:05 woosley left 10:10 grondilu joined 10:45 cognominal joined
diakopter o/ 11:04
tadzik o/ 11:08
jnthn afternoon o/ 11:16
diakopter
.oO( 240 km/h hurricane hitting India O_O )
 11:33
jnthn Yes. Ouch. :( 11:36
diakopter India's weather service says storm surge of 11 feet; US weather service says 30 feet. 11:37
jnthn Makes sense. US feet are bigger, like everything in the US is bigger... :P
tadzik what surprises me is that US weather service is still operating :P 11:38
jnthn Though, operating != getting paid to operate... 11:39
moritz but if the US has bigger feet, the number should be smaller
diakopter "The US navy's weather service reported winds at sea were gusting at 314kph."
moritz (but come on, 11 feet isn't much for the Pacific Ocean; you don't need a Hurrican for that)
diakopter
.oO( not too slow )
 11:40
Our winds are pi today.
TIMES ONE HUNDRED
nwc10 the US Navy has gone metric?
moritz 0.1 kpi :-)
grondilu has never heard of a wind speed over 300km/h on Earth 11:41
nwc10 is it feeling OK? Is this what a government shutdown does?
"Pay us, or we go metric!" 11:42
moritz let's not pay them until it sticks :-)
grondilu they received a promise to be paid retroactively once the budget is voted. 11:43
diakopter www.usno.navy.mil/JTWC/ 11:44
jnthn US. No!
moritz grondilu: that's, like, creating debt, isn't it? 11:45
grondilu well, a promise to pay is a debt, by definition. 11:46
moritz and I thought that was explicitly forbidding during shutdown
grondilu I guess thy made an exception. 11:47
*they
diakopter 138k people died in Myanmar in 2008 from a smaller cyclone.. 139,000 people died in Bangladesh in 1991 from a smaller cyclone. 300,000 people died in Bangladesh in 1970 from a smaller cyclone. 11:49
how many people is 1.2 crore people 12:00
12000000 12:01
airport & 12:02
jnthn hopes the most endangered folks will have managed to evacuate... 12:03
dalek arVM: 6b0e776 | jnthn++ | / (6 files):
Add missing bindhllsym op.
 13:10
jnthn To do to help Rakudo on Moar: port hllize, hllizefor ops (look at JVM impl) 13:23
FROGGS[mobile] Uhh, bindhllsym... nice 13:44
I've seen it complaining about that one when compiling moduleloader 13:45
jnthn: have you seen my note about uv_loop_delete? 13:46
jnthn FROGGS[mobile]: Yeah, but I'm not sure what to do about it... 13:49
diakopter FROGGS[mobile]: explain it more? 14:48
hm, now to see if master will merge into the gcorch branch 15:16
erm.. 15:18
no conflicts!!?!?
weird..
timotimo that means there's only non-straightforward conflicts :( 15:20
diakopter :_
well, the gcorch branch ...... uhm 15:24
bootstrapped nqp just fine
uhm.
where went all the errors I was seeing
uhm. 15:25
timotimo uh 15:26
dalek Heuristic branch merge: pushed 76 commits to MoarVM/gcorch by diakopter
jnthn diakopter: Maybe many of them were things fixed in recent weeks that were always lurking, and you were unlucky enough to hit 'em with your branch.
timotimo yeah, hit them with a stick! hit 'em with a branch!
diakopter well, all tests successful
so, uh 15:27
JimmyZ what?
jnthn Any speed improvement? I guess the better gcorch is mostly about making multi-threaded better though...
timotimo have you tried tuning your gc parameters into the "omg super paranoid find all errors" mode?
JimmyZ time to merge gcorch to master?
diakopter oh wait.
timotimo there the errors are now :) 15:28
diakopter forgot I raised the gc ratio to 5000
timotimo so basically "never run"?
diakopter well, 50
jnthn gen2 never run
Wants to be 10ish :)
diakopter but it runs gen2 on the shutdown 15:29
but no code runs after it
but...
anyway I'll try nqp test again
lowered it to 10; nqp test was clean :S 15:31
so, uh
timotimo building nqp, too?
diakopter sigh. 15:32
trying
dalek arVM/gcorch: 4b16365 | diakopter++ | src/gc/collect.h:
back to normal ratio
timotimo if you can't succeed to break something, give it to a little kid and tell it "don't you break it!"
that's also how you split atoms
jnthn
.oO( "nuclear family" )
diakopter yeah, built nqp from scratch and all tests successful 15:36
timotimo very nice! :) 15:37
jnthn \\o/
diakopter++
diakopter so I guess it's ready for review :S 15:38
jnthn ;)
diakopter just ignore the use-tc-pointer-as-owner-instead-of-thread-id
I know an efficient way to fix that sometime
jnthn ok
I guess for now it makes all the objects bigger?
diakopter but it's not strictly necessary except to save memory
jnthn *nod* 15:39
I can see how it makes things easier this way
diakopter also it renamed owner to manager since it was being used for two things - in which thead's memory is the object allocated, and which thread has a lock on the object in shared mode... and so I removed the second sense for now... 15:40
so for objects that can enter shared mode they'll need an owner field 15:41
I mean, theoretically we could derive which thread allocated the object by studying its address
but that would be pretty onerous 15:42
heading to gate 7 15:43
&
s/7//
timotimo the moarvm gc will always be "stop the whole world" even in multithreaded situations, right?
jnthn Yes 15:45
JimmyZ stop the world works in <<Matrix>> 15:46
the movie :P
timotimo the dejavu thing would seem more like the matrix people use STM for their stuff
diakopter erm 15:49
well I must've been doing something wrong
because it doesn't run at all. :)
okay, that's more familiar.
I guess it was using the old build...
:S
timotimo oops :)
diakopter ok really akf 15:50
akf
afk
jnthn good flight
17:41 ssutch joined 17:53 FROGGS joined 19:05 rurban joined 19:06 cognominal joined
diakopter jnthn: heh, the frame cache isn't being used at all and I'm not sure why 20:20
(landed)
I'm actually not sure any frames are being freed either. 20:21
timotimo bah. frames are cheap 20:27
jnthn diakopter: I was looking at a profile the other day that suggested we were missing it a bunch
diakopter: If we're not freeing them then we'll be using a hell of a lot of memory :P
timotimo also, if you throw away frames, what do you do with photos?
diakopter jnthn: and indeed we do use a lot of memory.
jnthn diakopter: Could it be related? Find out in this week's episode of Moar Work Needed! 20:28
diakopter mine is invoking like 400k/s
20:29 gshank joined
diakopter er 20:33
no
rhat's how many mallocs from frame invoke 20:36
timotimo 400.000 malloc calls due to frame entering? 20:38
per second?
jnthn Quite believable
And yeah, we'll be missing that frame cache.
That was a hot spot int he profile I did the other day 20:39
timotimo does the frame cache cache one frame to be re-used per ... frame-having-thingie?
jnthn arnsholt: omgz I just had win32-api-call.p6 work on JVM \\o/ 20:42
timotimo very cool! :)
jnthn un, mis-channel too :) 20:43
diakopter timotimo: actually it's like 70 per static frame or something 21:03
timotimo yeah, that'd do it :)
jnthn diakopter: I'm guessing some of those don't stay around though? Used during verificaiton? 21:07
diakopter jnthn: I think none get put on the cache list because none are hitting refcount 0 21:25
jnthn hmm 21:26
/* Decreases the reference count of a frame. If it hits zero, then we can * free it. Returns null for convenience. */ 21:27
says MVM_frame_dec_ref 21:28
Is that really wise, if we didn't actually reach 0?
Hm, maybe yes.
I assume the comment about MVM_decr returning what the value used to be is also true? 21:29
ah, looks like 21:30
#define MVM_decr(addr) AO_fetch_and_sub1_full((volatile AO_t *)(addr))
diakopter: oh, I think I see it. 21:32
/* Initial reference count is 1 by virtue of it being the currently
* executing frame. */
MVM_store(&frame->ref_count, 1);
But we never actually subtract that when it ceases to be the currenlty executing frame.
diakopter o_O 21:34
it's not the exiting decrement?
there in er, returning 21:35
timotimo moarvm is going to become even faster?! ;)
jnthn return_or_unwind doesn't seem to decrement returner. 21:36
diakopter it used to :P
jnthn Decrements caller. :)
diakopter hey, all I know is when I added the frame cache the man-or-boy improved noticeably without measuring 21:37
like 30-40%
timotimo did it also start giving wrong results? :D 21:38
jnthn Well, lemme try adding the decrement
diakopter no.. 21:39
21:39 cognominal joined
diakopter er, scratch that no 21:39
was answering tt
jnthn Blow me, I think it just build without even passing 200 MB of RAM.. 21:41
diakopter rejects the offer
jnthn OK, optimzied build time
diakopter: I, uh... :P
timotimo wasn't that at over a gig of ram before? 21:42
diakopter whom can we blame for the sandbagging
jnthn I think it was hitting 2GB
diakopter yep
2100MB
jnthn hah, now nwc10 might be able to pi it
bah, just blame me :P
diakopter heh. 21:43
timotimo why, that's amazing!
diakopter here I was optimizing 2 percent here, 5 percent there on the plane today
timotimo the last time i tried to build it, my computer froze up from the swapping. then i took a long nap.
diakopter jnthn: 5% of the time of the duration of the qregex test was spent in get_storage_spec until I refactored it to pass a pointer to an MVMStorageSpec... saved 5% 21:45
timotimo so you optimized away 100% of that time!
diakopter all that copying of structs on the stack
jnthn hah, -j6 now lets me run the NQP test suite in 9s
diakopter jnthn: try -j30 for fun 21:46
jnthn 8s
:P
21:46 benabik joined
jnthn Anyway, biggest memory usage I see anywhere during the build is 163MB 21:47
diakopter hahahahahaha
jnthn One downside: 24-modules.t segfaults now.
diakopter hee.
probably just exposing some other bug
timotimo that's a trade-off you'd gladly make if you're on a raspberry pi ;)
jnthn yeah 21:48
I'll try and hunt it
Well, that's an improvement :)
dalek arVM: a0c41ad | jnthn++ | src/core/frame.c:
Make sure to refcount-- frames we return from.
 21:49
diakopter jnthn++ 21:51
jnthn: was that the only one that segfaulted/ 21:52
?
jnthn diakopter: 24-module.t is only one that segfaults 'cus of the above patch, yes
diakopter: It does the rest, and builds NQP, just fine with debugging and optimized build 21:53
diakopter heh ok
jnthn I'm suspecting autoclose
diakopter I'm suspecting ->sc 21:54
jnthn no, that looks reasonable...
jnthn breaks out the debugger 21:55
diakopter I bet someone took out that returner decrement because it "fixed" bugs that were appearing... by keeping objects alive :)
jnthn oh...tricky 21:56
21:56 colomon joined
jnthn No, I think I see what's going on now I know where it explodes. 21:56
diakopter do tell
jnthn autoclose thinks "oh, we don't need to allocate any ->work area 'cus it's just an autoclose frame"
But then the frame hits zero and ends up in the frame pool
Which then assumes it has a ->work 21:57
(at the point of re-use)
diakopter easy check is add a fix
erm.
easy fix is add a check
jnthn right
diakopter same for env?
jnthn trying a patch 21:58
diakopter oh, also... 21:59
it was spending 4% of time in get_env_hash
or whatever it's called 22:00
jnthn huh? 22:01
diakopter MVM_proc_getenvhash
jnthn wtf...
diakopter 4% of the qregex time in that
jnthn Why, ooc?
That sounds like something we want to make it not do...
I'm guessing it's doing it a lot of times?
diakopter oh wow.
string_substring and string_index are SLOW 22:02
54 times
100 calls to each of those string ops per call
er
100 to substring
50 to index
timotimo While looking for 'QAST.moarvm': no such file or directory - *still* getting this error while building stuff ... ?!?
timotimo tries things
diakopter 50 to utf8_decode
timotimo i still need the moarboot branch, correct? 22:03
diakopter no 22:04
jnthn no, master
diakopter master
timotimo since when! 22:05
that may explain things :)
still telling me there's no QAST.moarvm to be found. weird. 22:06
i'll have to find out what's with that some day.
diakopter timotimo: need to realclean everything probably 22:07
jnthn timotimo: Can you gist me your moar and nqp build logs, so I can see what you're trying? :)
diakopter: Found our excessive getenvhash usage 22:08
multi method as_mast(QAST::CompUnit $cu, :$want) {
...
if (nqp::getenvhash()<MVMCCDEBUG>) { say($cu.dump); } 22:09
That seems useless since you can get that with --target=ast :)
diakopter hee 22:10
yeah that was supposed to go away long ago
jnthn OK 22:11
dalek arVM: 88c3f16 | jnthn++ | src/core/frame.c:
Fix case where we re-use a context-only frame.

It will be missing a ->work, which is easily rectified.
jnthn getenvhash does make the hash afresh each time, but I think it needs to
Otherwise tests will fail
Anyway, will remove the MVMCCDEBUG line :)
diakopter meh I'm sure there are ways to optimize it specially though if it's hot again someday
want some more hotspots? 22:12
jnthn Sure, though we may have new hotties after these fixes :)
diakopter pushing one
dalek arVM: d003447 | diakopter++ | src/ (31 files):
help the optimizer a bit by making get_storage_spec not use so much stack and copying
arVM: 9e54e55 | diakopter++ | src/core/frame.c:
Merge branch 'master' of github.com:MoarVM/MoarVM
arVM: f1e7917 | diakopter++ | src/core/frame.c:
Merge branch 'master' of github.com:MoarVM/MoarVM
diakopter feel free to revert it if you don't want to do that 22:13
but it saves a bunch for me
jnthn looking
I'm a little worried that the new thing on tc will be fragile 22:15
diakopter as long as get_storage_spec doesn't recurse..
timotimo gist.github.com/timo/8210a8e008c9941ad26e - that's my output 22:16
jnthn Well, or we don't write code that wants to talk about two different thing's storage specs at the same time...
It's the latter one I worry about
Thing is...
diakopter ah.
jnthn I think I know a more efficient way to solve this... :)
diakopter ok how
jnthn For many of the REPRs, storage spec is always the same 22:17
diakopter my other idea was to pass pointers to the locals of values we wanted, and for the get_storage_spec to check each one, and if it's not NULL, populate it
jnthn So we just have a static MVMStorageSpec and return a pointer to that.
We change get_storage_spec to always return MVMStorageSpec *
So there's not copying OR setup work
diakopter 99.99% of the time was from the p6opaque one... oh wait 22:18
jnthn For things that *do* want to customize it, they want to do so by type
So we can just allocate one hanging from, say, P6opaqueREPRData
And just return the pointer to it
Then free it in free_repr_data
diakopter oh wait.
it wasn't the copying
bleh
ok revert it :) 22:19
jnthn I agree we should change this, just not like this...we should change it so it returns a pointer to something, then set it up one time and return the already set up thing :)
That should be a bigger saving
And not make me worry about the new thing off tc ;)
22:21 colomon joined
diakopter right, like I said :P 22:22
jnthn Shall I do the revert?
diakopter yes plz.... .oO( making me grovel... ) 22:23
timotimo Routine declaration requires a signature at line 4104, near "(MAST::Nod"
this is the real error, btw
diakopter anyway 6% of time is spent in get_attribute 22:24
because there are no slot hints
jnthn :)
I noticed that a load of those are related to invoke btw
When we do the lookup of the code ref in a code object 22:25
Fixing that would probably help avoid a bunch of the more costly get_attribute calls
diakopter no
these are all direct from the interpreter
jnthn oh 22:26
diakopter I'm looking at the call tree only
not the aggregate
jnthn OK, the profile I did of QAST.nqp compilation showed the path through invoke to get_attribute also has the no-hint issue
diakopter 5% of time is spent in at_key_boxed, in the uthash memcmp entirely 22:27
dalek arVM: 535fd59 | jnthn++ | src/ (31 files):
Revert "help the optimizer a bit by making get_storage_spec not use so much stack and copying"

While something in this space is helpful, this isn't the ideal way, as discussed on #moarvm.
diakopter 5% of time is spent in at_key_boxed, in the uthash memcmp entirely
jnthn hm 22:28
Well, that's kinda believable
diakopter 740k calls in 5 seconds
jnthn I guess it already has somewhere the "if the memory addresses are the same, we need not compare" opt?
oh, memcmp probably has that...
And it is probably hit less than I would think... 22:29
diakopter that's just direct from interp
4m calls to mvmarray at_pos 22:31
(total time 1%)
grondilu one thing I don't understand though. The first stage won't go in orbit, so it won't make a whole turn aroung Earth. Therefore when it renters, it will be quite far away from the launching area. Will it have to travel on this thanks to its propulsion? 22:48
timotimo that's why it can go horizontally 22:49
grondilu oops, that was the wrong channel, wasn't it? 22:50
timotimo yep
grondilu reposts in #space
jnthn moar channels create moar confusion...
diakopter jnthn: what blocks getting the attr slot hints? 22:55
jnthn nothing afaik
just needs doing
diakopter well, what needs doing about it
also, what's needed to get the method cache
jnthn making sure nqp::hintfor is implemented 22:56
And then using it for QAST::Var node compilation
Just passing the hint to the getattr and bindattr instructions
diakopter 400k calls to find_method took 800ms 22:57
jnthn The "method cache" is really just a hash
The smarter thing is likely part of the specializer
oh btw...I had an idea on that stuff 22:58
I realized that we can use the AST we build for the specialization stuff not just to JIT, but potentially also to spit out refined bytecode that includes some "secret"/unsafe instructions (ones that you can never get past validation). 22:59
diakopter yah 23:00
jnthn Meaning I can play around with it without having to worry about the code-gen side of JIT and still get some performance wins as a result and a good way to make sure the "safe" things really are safe :)
Anyway, that ties into what you just asked in so far as, if we type-specialize and we know that the method we found was produced out of the method cache hash, not dynamically, then we don't need to look it up :) 23:02
diakopter yeah but find_method is only 2%
23:02 woolfy joined
jnthn Sure, but it all adds up :) 23:03
Plus that is the step before going ahead and doing inlining of said method, if it looks worth it
diakopter okay, I ran a new profile 23:13
33 seconds of qregex 23:14
roughly 1 million interpreter operations/second
jnthn That's while profiling yes?
diakopter yes
so probably a 10x slowdown 23:15
jnthn Was gonna say, t/qregex runs in less than 33s here :)
Right, that oom
diakopter jnthn: i.imgur.com/gnaHCgK.png 23:17
zoom in
wow. 23:18
only 1 gc run per second average
that's... not much.
jnthn That number of calls in MVM_interp_run is odd... 23:19
diakopter it's counting each switch loop for some reason 23:20
jnthn ah
diakopter almost as if it recognizes teh pattern as a tailcall-ish
jnthn This is instrumented, yes?
Not sampled?
diakopter yes
I unchecked "skip small functions" 23:21
er, exclude
the gc_free that is taking 4.29% is the one in mvmcode 23:22
jnthn Probably 'cus it decrements an outer and then ends up shoving stuff back into the frame pool 23:23
diakopter I'll look 23:24
jnthn MVM_sc_get_object is a bit hotter than I'd expected 23:25
Though I'm not worried about it
diakopter actually no, 3.6% of the 4.29% are calls to free after decrementing
jnthn I know once we JIT those will go away
diakopter which means the cache chain could use being longer :)
heading out & :) have fun 23:26
jnthn you too o/ 23:27
23:30 grondilu left
jnthn sleep & 23:39

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!