github.com/moarvm/moarvm | IRC logs at irclog.perlgeek.de/moarvm/today
Set by moderator on 27 September 2013.
01:24 benabik joined 01:45 FROGGS_ joined
diakopter o_O running ..\\moarvm nqp.moarvm t\\nqp\\19-file-ops.t takes 478 ms with --debug --optimize --profile 04:12
for some reason visual studio's profiler is telling me how many times it looped through the interpreter switch... o_O 11,069,894 04:13
12 gc runs 04:14
11% of time spent in the gc
15% of time spent allocating, including gc runs 04:15
whoa!
almost the entire time in the gc is spent in MVM_gc_mark_collectable 04:16
that's... unexpected (until I go look in the code)
well, hm 04:18
HUNH. 04:23
of mvmarray.c, p6opaque.c and mvmstaticframe.c, each had a single call to gc_mark that took >150ms... which was >98% of the total time spent in that function in the program run 04:24
TimToady maybe they were having a concert or something
diakopter hee 04:25
.tell jnthn interestingly, after optimization, only 0.04% of time is spent in the compunit loading... o_O 04:31
yoleaux diakopter: I'll pass your message to jnthn.
diakopter 9% of time is spent in msvcrt startup o_O 04:33
loading windows dlls and such
9% of total time spent in mvmarray.c push function! 04:35
exclusive time!
14% of program time is spent in exclusive time in mvmarray.c 04:36
ewww
1m calls to at_pos in there
O_O
oops, no, 16% of program time in mvmarray.c 04:37
wow. 04:38
fully 28.42% of time was spent in interp.c 04:39
not very informative, that.
1m calls to memcmp /o\\ 04:40
400k calls to memset; 223k calls to memcpy
almost no time doing hash-related stuff.... o_O 04:41
<2% of time in string-related stuff
so... arrays are super-heavy
and the interpreter... gotta get rid of that ;)
<1% spent running NFAs 04:43
TimToady: i.imgur.com/RCvoa0E.png 04:50
TimToady: anything stand out to you? 04:56
JimmyZ: ? :)
nwc10: oy 04:57
ingy o/ 04:58
diakopter ingy: oy
oops sbux closing &
. 06:28
moritz .. 06:35
06:48 ssutch joined 07:10 FROGGS joined 07:42 foo_bar_baz joined 08:09 ssutch joined 08:21 Woodi joined 09:46 cognominal joined
diakopter jnthn: get this... if I raise the ratio to 50, no errors :/ 09:48
jnthn diakopter: of course, you just never reach a gen2 collection...
diakopter t\\serialization\\01-basic.t ............ QAST::Block with cuid cuid_29_1380793736.057255 has not appeared 09:49
well it does say that
jnthn :S
diakopter (maybe that tests runs the gc 50 times!)
jnthn: sure enough, the gc runs 52 times :) 09:54
jnthn: I've noticed it fails usually on the 2nd run after the gen2 run
er, after the 2nd run after the gen2 run 09:55
though sometimes after the first one after the gen2 run
any ideas?
jnthn Curious...that makes it sound more invovled that simple wb issue
I wonder if something is wrong with the "I refer to the nursery" list cleanup 09:56
diakopter ah, the one part I haven't gone over with a fine-toothed combpickroll
jnthn MVM_gc_collect_cleanup_gen2roots 09:57
09:57 lizmat_ joined
arnsholt diakopter: i.imgur.com/7s0ALeF.gif ? =) 09:58
diakopter arnsholt: exactly. ludicrous speed. 09:59
arnsholt As long as you don't hit plaid 10:00
diakopter wonders if jnthn has seen that movie
jnthn no :) 10:01
Or I don't get the reference, at least... :)
arnsholt It's Spaceballs 10:08
Very funny!
10:32 Woodi joined 11:01 FROGGS[mobile] joined 14:15 jnap joined 14:41 larks joined
FROGGS /home/froggs/dev/nqp/install/bin/nqp --target=pir --setting=NULL --stable-sc --output=NQPCOREMoar.setting.pir nqp-src/NQPCORE.setting 14:57
Confused at line 621, near "IO Methods"
:o(
is this due to cursless? does somebody know?
jnthn No idea 14:58
14:58 benabik joined
FROGGS k, will bisect it I think 14:58
ohh, that is inside a pod block 14:59
jnthn oh...eeks
tadzik sounds like a job for a pod man 15:00
FROGGS *g*
... testing 15:04
okay, 2013.09 is fine 15:12
bisecting told me it is: github.com/perl6/nqp/commit/9ee3ed...9a62517e77 15:48
jnthn OK 15:49
That probably means NQP::Grammar is making a bad assumption somewhere...
And we got away with it before this commit
I don't think the commit itself is wrong, even if it's "to blame" for the regression 15:50
FROGGS hmmm
same problem on nqp@jvm fwiw 15:51
I'll dig into this
jnthn Great :) 15:52
TimToady 59.4 to 55.0 seconds, so more cores may be a factor; alternately, it throttles them harder when it's getting hot :) 15:58
jnthn Still, a welcome little improvement.
TimToady oops, ww :) 15:59
16:13 not_gerd joined
not_gerd o/ 16:13
diakopter o/
16:14 ssutch joined
not_gerd so, I've done some thinking on how to unify my DLL and ctypes work and get it into a shape that's more amenable for eventual merge 16:14
I probably won't have time to work on it until the weekend 16:15
should I write something up about what I'm planning to do for $interested_parties?
jnthn not_gerd: I'd rather keep the two apart
We need extops as soon as work on the Rakudo port starts. 16:16
The ctypes stuff I'd rather not have to worry about until later.
not_gerd I'm actually planning to make ctypes my test case for VM extensions ;)
jnthn OK, but still, please make it so I can review/merge extops on its own. 16:17
not_gerd sure
jnthn thanks :)
not_gerd the thing is, a DLLSym is just another type of pointer 16:18
if factored right, there's potential for unification
diakopter so use one from the other 16:19
you can make thd ctyoes branch vranch from extops vranch
not_gerd oO( those pesky off-by-one errors ) 16:20
diakopter :D 16:21
jnthn vranch!
timotimo salad with vranch dressing? 16:35
FROGGS jnthn: it is this branch btw: github.com/perl6/nqp/blob/master/s...qp#L90-L95 16:45
jnthn FROGGS: oh... 16:47
identifier is now declarative
.*? is also I guess :)
FROGGS and since it is declarative... ? 16:48
diakopter mmmm vranch 16:49
jnthn: did you see my various metrics from last night pasted here? 16:50
jnthn FROGGS: It will eat anything
diakopter: Glanced over them this morning...I should re-read 'em :)
diakopter mainly just see that imgur link 16:51
infer what you would
jnthn FROGGS: It'll come out longer than begin, for example
FROGGS: Because begin will match identifier, and .*? will match whatever comes after it. 16:52
FROGGS: In fact, if you do .*? then the LTMer will eat the whole document I suspect :P
FROGGS: Anyway, my guess is we never hit the 'begin' branches any more
FROGGS: Try a {} after <identifier> 16:53
FROGGS does so
diakopter . in a token eats whitespace? 16:55
hm, I guess
jnthn Sure 16:56
. is everything :)
diakopter yeah but..
what's non-whitespace 16:57
\\W ?
jnthn \\S
diakopter er, duh.
jnthn ok, dinner time...
jnthn hands diakopter another coffee :)
FROGGS dinner &
diakopter feels the coffee fall out my toes
jnthn: mostly I realized there was almost 0 gain to be had by optimizing deserialization of strings like we talked about 17:00
.. unless someone already did it while I wasn't looking
18:19 ssutch joined
FROGGS hmmm 18:31
diakopter ?
FROGGS ../moarvm nqp.moarvm --target=mbc --output=foo.mbc t/nqp/01-literals.t 18:32
MVMHash representation requires MVMString keys
it gets an MVM_REPR_ID_P6str instead
diakopter oh, it could learn to unbox I guess
FROGGS ... of an MVM_REPR_ID_MVMString
diakopter er 18:33
TimToady
.oO(vranch roast coffee)
diakopter the compiler should have it unbox the keys before passing it to the hash add
FROGGS diakopter: this happenes when calling exists_key with an P6str 18:34
diakopter yeah; whatever compiler is outputting that qast needs to unbox the key
FROGGS k... 18:35
ahh, it happens with ../moarvm nqp.moarvm --target=mbc --output=foo.mbc -e '' too 18:36
so it is not even the test file
diakopter look for exists_key in the qast compiler
erm, I think.
jnthn should decide whether he wants the hash ops to unbox or whether the compiler should inject the unbox operation 18:37
timotimo .o( that throws a vranch in the machine ) 18:43
diakopter lunchish& 18:51
FROGGS why is it called at_key_boxed when the to-be-passed key must be an MVMString? 18:56
timotimo maybe it returns to the return value? 18:57
FROGGS returns to the return? 18:58
refers perhaps?
timotimo yes, refers to the return value's type 19:00
or kind or mode or whatevs.
FROGGS hmm, if it is so I don't see it 19:01
timotimo was just guessing 19:02
FROGGS I don't see any boxing at all tbh
but that might be my very own problem :o) 19:03
okay, it gets called by reprconv.c MVM_repr_exists_key 19:07
diakopter a P6Str is a box 19:15
19:15 not_gerd joined
not_gerd FROGGS: see documentation in 6model.h 19:15
diakopter not_gerd: :P 19:16
that's not terribly helpful :P
FROGGS not_gerd: thanks
not_gerd well, it confirms that the boxing indeed refers to the result type 19:17
diakopter
.oO( is there documentation there? )
oh...
jnthn It's about the result type, yes
not_gerd it of course doesn't help with the problem of how it should handle its arguments
diakopter jnthn: should the hash ops unbox the key?
jnthn No, they should be taking a str reg, and do afaik 19:18
I think the error is from within the bowels of the MAST assembler...
diakopter hrm.
jnthn So it's something in there calling it without doing an appropriate unbox, I'll bet
diakopter actually
FROGGS I can only trace it to src/6model/repronv.c:162 19:19
but I can't find a call to MVM_repr_exists_key with a REPR id != 0
diakopter jnthn: actually, it's a runtime error, and the keys take objects
how do you know it's exists_key 19:20
FROGGS printf
diakopter oh
jnthn: no clue why the keys take objects
jnthn Look at the stack trace
FROGGS and this error message appears only once
jnthn at nqp-src\\QASTMoar.nqp:7958 (./QASTMoar.moarvm:assemble_to_file:4294967295)
19:20 jnap joined
diakopter er, I don't remember 19:20
jnthn Then at what is there
method assemble_to_file($mast, $file) {
__MVM__masttofile($mast, self.node_hash(), $file)
}
That's the thing that calls down to the MAST -> bytecode assembler. 19:21
diakopter okay; the error message is in MVMHash.c
jnthn Sure, that's called by the assembler 19:22
existskey doesn't show up too many times in there...hmm
bah, easier to look at it under the C debubber 19:23
bugger
debugger
lizmat hehe
diakopter FROGGS: existskeyop calls exists_key
existskey op
jnthn It's in get_string_heap_index 19:25
FROGGS I added a print statement before every call to MVM_repr_exists_key, and none of them got hit
jnthn in src/mast/compiler.c
diakopter erm.
jnthn strval there is not what it should be.
I hope we're debugging the same bug here?
FROGGS ahh, there
yes, seems like
the line of the stacktrace you pastes is exactly like mine 19:26
pasted*
diakopter FROGGS: lots of times it calls exists_key diretly
FROGGS diakopter: that might be the reason why my method didn't work out *g*
diakopter yeah the convenience routines (and macros) were added relatively recently, and not all calls converted for speed reasons 19:28
(paranoid superstitious speed reasons)
jnthn This is odd... 19:29
It's in a code branch that should only ever have a MAST::SVal
and
class MAST::SVal is MAST::Node {
has str $!value;
That's a native string slot
diakopter sounds like a critical bug somewhere :)
jnthn yeah 19:30
quick trip to the shop; bbiab 19:31
diakopter is annoyed I have to nmake realclean the cross compiler when switching branches 19:54
because of some missing Makefile dependencies
19:56 japhb_ joined
jnthn back 20:03
diakopter At Frame 304, Instruction 181, op 'shell' has invalid number (2) of operands; needs 4. 20:10
argh. nmake realclean didn't even fix it
FROGGS shell() gets the cwd and env now too since a few weeks... 20:12
maybe something is just outdated?
diakopter right, that's what I said 20:13
FROGGS so we agree?
:P
diakopter no, but I have no idea what
I wish there were a git status command that ignored the .gitignore
*was 20:14
not_gerd just a guess, but what about re-running update_ops? 20:15
jnthn diakopter: git status --ignored 20:18
So, lemme look at this mbc bug a bit more... 20:24
dalek arVM/dll: be0fe24 | jnthn++ | src/core/interp.c:
Fix STable claiming.

With this, t/serialization/01-basic.t now fully passes.		 20:29
not_gerd for got to push those yesterday
20:29 dalek joined
jnthn diakopter: So far, discovered that somehow an SC ends up with its ->handle pointing to something not an MVMString. 21:05
diakopter oopsie
jnthn yeah 21:06
but struggling to see where
createsc ends up with the Right Thing
diakopter hrm 21:08
perhaps the quasi-weak hash
nah
jnthn it's ok in MVM_sc_create
diakopter oops MVM_HASH_GET(tc, tc->instance->sc_weakhash, handle, scb); 21:09
oh, nm :/ 21:10
jnthn yeah, added a check there and it seems fine too
diakopter well, what *is* in that slot 21:11
if not a string
junk pointer?
jnthn A P6str 21:12
diakopter must be from deserialization
well wait, what string is in the p6str 21:13
what content
21:14 donaldh joined
diakopter does the P6str have a sc? 21:14
jnthn oh, duh...I did something silly 21:15
wait a mo
diakopter ok
jnthn yeah, false lead
We *are* somehow getting an object on a string reg 21:16
But not there
diakopter yikes :)
could theoretically put a very inefficient guard in MVM_ASSIGN_REF :)
'course, that may never terminate... 21:17
diakopter goes to try to repro 21:24
.. instead of just imagining... 21:25
oh yeah, I can't build the cross compiler
argh.
21:28 foo_bar_baz joined
FROGGS diakopter: what is your nqp --version? 21:31
diakopter ARGH 21:32
rogue zombie nqp.exe
FROGGS :/
diakopter I always forget to check that.
PerlJam wonders if "zombie nqp" is the beginning of the zombie apocalypse 21:33
not_gerd bye, #moarvm 21:36
diakopter o/
21:37 not_gerd left
diakopter jnthn: re object in string register, I knew I put that redundant repr check in for a reason.... ;) 21:37
jnthn I'm struggling to reproduce it in a small test case 21:38
diakopter I think it'll be easier to try to trace it back by logging in giant hashes of addresses... :D :D :D 21:40
(I'm only half-kidding!) 21:42
I've wanted such a feature already 21:46
though it's replicating address sanitizer, some 21:47
jnthn this is *really* confusing... :S 22:01
diakopter well that's worrisome 22:02
any learnings?
jnthn Well, the fail starts here: 22:03
QAST::Op.new( :op('createsc'), QAST::SVal.new( :value(nqp::scgethandle($sc)) ) )
FROGGS should be :s for a natively typed str
diakopter jnthn: oh you know what
jnthn If I put something in P6opaque that makes sure that when you bindattr_s, it really is a string, then it explodes 22:04
in setting $!value inside QAST::SVal
diakopter MVM_sc_get_handle is actually wrong
jnthn oh?
diakopter yeah it needs to do the resolution check like the other sc_get things
jnthn um 22:05
diakopter er, hang on; I'll tell you what I mean
jnthn like *none* of the other things do in that file?
diakopter se
sec
:)
erm, where'd it go 22:06
diakopter keeps looking
whaaaaaa 22:08
where'd it go :S
I swear I wrote a few lines of code to fixup the sc body of ones once they're resolved 22:10
and intended that check to go in all the accessors
oh, it's in MVM_sc_get_sc 22:11
cu->body.scs[dep] = sc; 22:12
oops, that needs a MVM_ASSIGN_REF
probably not the problem tho
also it should null cu->body.scs_to_resolve[dep] 22:13
ergh
erm 22:14
don't listen to me
*today 22:15
*except for this statement
jnthn: lemme take this on; I have a decent idea how to tackle it, at least, I think 22:17
but not at the moment 22:18
but this evening
FROGGS it is already past midnight here, so you might get the chance to tackle it 22:19
jnthn diakopter: ok 22:27
diakopter: The compiled output looks very sane, fwiw
diakopter compiled output of what? 22:28
bytecode dump of something?
jnthn yeah, --dump
diakopter of nqp.moarvm? 22:29
jnthn QASTMoar.moarvm
The thing that calls scgethandle
diakopter oh :)
bye bye dalek 22:33
22:33 dalek joined
FROGGS well, good that dalek survived in our commit-channel :o) (#parrot) 22:35
jnthn diakopter: Anyway, I won't hunt this any further for now. 22:40
Something's rotten, somewhere
diakopter I'm guessing it's something small 22:44
jnthn: how did you conclude an object was definitely getting into a string register
s/conclude/determine/ 22:45
jnthn Let me give you my patch...
diakopter: gist.github.com/jnthn/6818287 22:46
diakopter :) string is not
jnthn aye 22:47
Anyway, that triggers in the place I pointed to
The reason we don't hit it without --target=mbc is 'cus we don't emit deserialization code.
If you don't put that it then we have the P6str object shoved in the str attribute, and it makes it all the way until MAST assembler, where it explodes (which is the original error we were seeing) 22:48
diakopter hrm 22:50
jnthn So anyway, that patch makes it blow up closer to the thing to blame.
I put similar in the scgethandle op, but to no aveil 22:51

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!