00:46 zakharyas joined
timotimo hm. adding the filename to spesh/dump.c output makes it more helpful, but it also leads to very long lines 01:01
now what would be really nice is to find a filename + line number annotation in the original and dump that 01:04
01:08 zakharyas joined 01:09 colomon joined
timotimo i'm not quite sure how to turn a hllboxtype or bootintarray operation into a set-like instruction ... set is only for moving data between registers, sp_get_o is for getting an object that's pointed at from a different object ... 01:16
i know i should be able to just set KNOWN_VALUE and value.o, but that could easily explode 01:17
theoretically i could just const_i64 it ... except on 32bit systems ... this is A Bad Idea™ 01:22
01:30 FROGGS_ joined 01:47 ilbot3 joined
dalek arVM/spesh_hll_and_boot_types: 07b4b07 | (Timo Paulssen)++ | src/spesh/optimize.c:
sketch out spesh for boot* and hll* operations
01:47
timotimo well, as expected just setting the known type + value and removing the instruction fails hard 03:06
oh my, why the F am i still up so early in the morning? 03:13
07:30 woolfy joined 07:31 woolfy left 08:24 lizmat_ joined 08:37 brrt joined
brrt \o 08:51
jnthn timotimo: Um, but facts.c already sets known type for all those.
timotimo: And if the instruction really isn't needed then dead instruction elimination will already get it 08:52
timotimo: And the JIT will typically turn the things into constants...
o/ brrt 08:53
timotimo: So there's not much more to do for them.
brrt constants? i didn't hear no nothing about constants?
or are we talking about wvals
jnthn Any object literal that's in gen2
brrt oh yes 08:54
jnthn Yes, same thing as for wvals
brrt that may have not been totally implemented yet
jnthn Ah, ok
brrt for strings it has
i had an idea that i'd like to implement 08:57
basically, we can add in the oplist file some information that gives us the name of the symbol as well as the parameters that should be passed 08:58
i.e. i could append something like &MVM_exception_die(iTr1a0), where everything between & and ( is the symbol name, and the string describes the parameters 09:00
hmm
oh, and after that comes the return value in some sense
09:01 FROGGS[mobile] joined
jnthn ah, to generate a bunch of graph.c instead? 09:17
(Instead of hand-maintain it, that is...) 09:18
brrt nods
or well, even to do it dynamically
jnthn Comes after getting moar-jit merged, I'd say :) 09:26
But yeah, could be a nice idea.
dalek arVM/moar-jit: e95a0b8 | (Bart Wiegmans)++ | src/jit/graph.c:
Split pre- and post- processing of instructions in jit graph

This will ensure that even as some instructions consume multiple ops, annotations will still be applied correctly.
09:28
brrt i'm a bit puzzled as to how MVM_frame_find_contextual_by_name really works with regards to inlines 10:06
jnthn With difficulty...
brrt i.e. it seems that it only works for invoking frames because it requires return_address
but it is initialized with the leaf frame 10:07
brrt can see that at least :-)
jnthn Yes. And we disallow inlining of getlexdyn 10:08
So the other case needn't be handled.
Plus getlexdyn doesn't consider the currently executing frame, iirc.
Since we know what lexicals are in the curernt frame, so my $*FOO; ... $*FOO := 42; in the same frame just compiles into normal lexical access. 10:09
brrt hmm
ok
so this is a bug that's not a bug because it's guarded by two other constructs
:-)
jnthn So yeah, we kinda get away with that case. You're understanding it right, though. :)
brrt nice
ok, i'm going to re-enable getdynlex and see if it blows up 10:10
fingers crossed
dalek arVM/moar-jit: dc2fca5 | (Bart Wiegmans)++ | src/ (2 files):
Support dynvar caching again

This was disabled because the JIT obfuscates where we are in the process, and thus we couldn't find exactly in which inlined frame we are. The JIT now keeps track of inline borders and so it is possible to enable this again.
10:11
brrt is amazed at how this keeps working 10:15
gist.github.com/bdw/79da1fdbf5ec7f17b593 new bail log with getdynlex support 10:19
dalek arVM/moar-jit: 3c923a6 | (Bart Wiegmans)++ | src/ (2 files):
Re-enable getdynlex
10:20
jnthn lexoticresult tends to come quite late on and so would win us a bunch of frames
paramnamesused tends to come early on so likely hides other things :) 10:21
brrt yeah, i know what's needed for paramnamesused, and i'm still in the pondering phase of that 10:24
:-)
also, lexoticresult can / should be implented as a primitive i think 10:25
hmm 10:27
it need not be
but it's just as well
MVM_exception_throw_adhoc never returns, does it? 10:33
jnthn Hm, looks like lazily deserializing method caches - even without the work to less eagerly deserialize/set code objects - saves another 8MB or so off the base memory for Rakudo.
brrt: Correct; it lngjmp's back to the interp
brrt good 10:34
:-)
ah... newlexotic is wrong 10:38
subtly so
brrt brb 10:44
11:23 brrt joined
timotimo ah, i knew i already saw code to handle boot* and hll*type >_< 11:36
that's why you don't code at 3 in the morning
brrt back 11:39
timotimo o/ 11:43
jnthn: i only got the idea to put these into optimize.c because i saw lots of those in a spesh log 11:45
so the dead code elimination doesn't kill them yet
may want to decrease the usages of the type for some ops in optimize.c
jnthn timotimo: Yes, in the past we were more hesitant about ->usages-- because deopt boundaries weren't handled properly. They should be now. 11:47
timotimo ok
also: holy crap, there are *many* useless set ops in some pieces of code 11:48
jnthn That's 'cus we turn various things into set :) 11:50
brrt uhm, maybe 24717904 is a bit large for an offset 11:53
jnthn much offset
brrt very so 11:54
dalek arVM: 4492a63 | jnthn++ | src/ (6 files):
Lazily deserialize method caches.

Saves another ~8MB off base memory for Rakudo.
timotimo i may need to help friends move a bunch of boxes before tackling the usages-- thing 11:56
brrt will the boxing be in time? 11:58
timotimo :)
12:05 colomon joined
brrt wow, offsets are really large 12:06
oh, i see why 12:09
they're transformed into bb pointers
that's
inconvenient
i'll be needing the instruction in some cases and the basic block in others 12:10
jnthn Why do you need those, ooc?
Those offsets would correspond to the unoptimized bytecode...sometimes. 12:11
(As in, not if the subgraph is inlined)
brrt the newlexotic op takes the hard coded bytecode offset
jnthn Oh...
brrt watchpoints are useful by the way :-o 12:12
jnthn And then uses it to resolve a handler, iirc?
brrt es
yes
jnthn wonders if we can't pre-resolve that when JITting...
...or at least use a label address 12:13
Since we have handler labels too
brrt hmmm 12:14
yes
that's possible too
jnthn That feels less fragile
brrt but i'll have to modify the newlexotic function a bit 12:15
i can even use the label number, how about that
timotimo it seems like i was just tripped up by the before/after refering to the logging instruction insertion, not the spesh process itself
it seems like boot* and hll*type are kicked out properly already 12:16
jnthn ah, cool :) 12:17
brrt: Yeah, or just a newlexotic_jit function :) 12:18
brrt oh, yeah, that's possible too
timotimo MVMArray only speshes create, i think it'll be easy to spesh things like elems as well
jnthn yeah, elems should be easy 12:19
Other things less so
timotimo aye.
brrt yay, copy paste coding :-) 12:20
jnthn: it occurs to me that after codegen, we might also restore the ins_offset to its then-current value 12:24
or wait
not
jnthn After code-gen we often throw the graph away... :)
brrt yeah, but not before the JIT has access to it 12:25
but i don't like that hack
jnthn no, that'd be horrible 12:26
timotimo huh. 12:27
for sp_get_i i should supply something like offsetof( MVMArray, body.elems ), right? 12:28
because that's what i'm doing right now and it faceplants pretty loudly 12:31
jnthn Something like that, yes 12:33
timotimo huhs
jnthn How does it fail?
timotimo Cannot call method 'MATCH' on a null object 12:34
at gen\moar\stage2\QRegex.nqp:552 (src/vm/moar/stage0/QRegex.moarvm:CAPHASH:4294967295)
jnthn Odd
timotimo maybe i'm trying to get the elems count from an array that's actually VMNull? 12:35
but wouldn't the non-spesh'd code have blown up in that case, too?
except if it would have caught the resulting exception
but no, we're relying on known_type, which would force the register to have non-null contents, right? 12:36
jnthn Should do, yes
I guess that the opt does come in with truthiness checks too
So it's possible there's an elems check 12:37
And that control access to something else
timotimo there is only a single sp_get_i in the resulting spesh log, which is inside !cursor_capture 12:38
the offset is 24, which should be the exact length of an object's header 12:39
we guardconc directly before that get_i
and afterwards we seem to push_i the result from elems somewhere
aye 12:41
cursor_capture does push_i($!bstack, nqp::elems($!cstack))
brrt hmmm
timotimo (and just to be sure i verified that it succeeds if i don't spesh it) 12:42
(don't spesh it == MVM_SPESH_DISABLE)
and without the elems optimization it succeeds, too 12:43
jnthn You are checking its known concrete as well as known type, I guess? 12:51
I guess that goes for all repr things...
timotimo oh! 12:52
the repr guard only checks for known type actually
that could be a problem
another thing we should fix soon is that the optimize_repr_op does "use_facts" for us on the type fact, but the spesh slot of the repr doesn't signal whether or not it actually did anything 12:53
so the use_facts ought to move into repr's spesh methods
i can do that, too
jnthn True
timotimo checking that FACT_CONCRETE is set doesn't fix the problem, unfortunately 12:54
brrt and.. i know why it fails, too 12:55
(this is something else entirely)
timotimo that's still good for you :)
we can just leave out the spesh for MVMArray's spesh, but we can't really leave out a big chunk of the jit like handler support ;)
AFK for a bit 12:56
brrt :-) 12:58
completely offtopic: my qualified biologists opinion on anti-aging and calico in particular is that it is the dumbest thing ever 12:59
all organisms age, because all organisms must tradeoff between reproduction, growth, and maintenance 13:00
that doesn't change when you throw google-type money against the problem 13:02
jnthn Trying to "deal with death" is a human obsession since forever, I guess; it's just that now we're seeing folks try to throw science at it instead of religion. :) 13:05
brrt i suppose that's true 13:07
but it's also incredibly naive about how organisms and especially large vertebrates actually work
and that's annoying 13:08
xiaomiao brrt: and if you reduce the "death" mechanism you amplify the "cancer" mechanism, most of the time
I'm not sure that's a good idea :) 13:09
brrt yeah, that is very true
very very true
it's a problem
:-)
despite that, progress on cancer has been pretty good the last few years, so i may be totally wrong 13:10
xiaomiao yes. I'm sometimes surprised by the progress in medicine 13:11
dalek arVM/moar-jit: 57b8266 | (Bart Wiegmans)++ | src/ (6 files):
Fix newlexotic

Newlexotic normally takes the bytecode offset of the frame handler goto label, but this is transformed into a pointer to the basic block during spesh graph creation. Instead of looking up the correct value using some hack, pass the label instead.
Because the goto annotation is attached to an instruction and the instruction points to a block, the labels of these would diverge if the annotated instruction was not the first of the basic block, as would happen if there where PHI nodes before. So I've changed the labeling code to disregard PHI nodes when labeling an instruction.
xiaomiao appendix removal with microsurgery and such tricks
brrt bbiab 13:13
jnthn will give latest moar-jit a spin after the current block/code object work he's doing :) 13:14
brrt please do 13:15
i've tested it of course, but more is better wrt to testers :-)
13:15 brrt left 14:17 brrt joined
brrt \o 14:18
timotimo ¯o¯
brrt :-)
much nqp / rakudo code churn 14:34
timotimo really quite looking forward to the next commit jnthn is likely going to push 14:35
brrt does the 'Cannot look up attributes on a type object' bug say anything to any of you? 14:36
timotimo oh?
haven't seen that come up in builds often
brrt ugh 14:37
my code is broken again :-(
jnthn timotimo: Turns out that it's not an immediate big win. 14:38
timotimo oh
OK
jnthn Which probably means we've another source of object "leaks"
timotimo sure; sadly we don't have a good tool to find those 14:39
but i'm still looking forward :P
jnthn Only me.
timotimo hah
jnthn
.oO( crap, did I just call myself a tool... )
brrt heard it 14:40
jnthn One good bit of news though; it does knock a little more off setting compilation
brrt hmm
timotimo oh? how does it do that? %)
brrt my settings compilation has risen to 62s by now
and
14:40 avar joined
brrt breaks 14:40
jnthn Well, just because it's cheaper to stick a pair of IDs on a MAST::Frame that generate a bunch of QAST, and then MAST, that does a wval lookup and so forth.
timotimo ah, yeah 14:41
that does make sense :)
14:42 ggoebel1111113 joined
dalek arVM: 34a1a35 | jnthn++ | / (9 files):
Enable code objects to be lazily deserialized.

We now can attach the indexes to a MAST::Frame, and keep them around until such a time as we need the code object. The goal is that, in conjunction with some other improvements, we might be able to avoid deserializing many code objects in Perl 6, along with signatures, parameters, etc.
14:44
nwc10 brrt: yes, I get "Cannot look up attributes in a type object" too.
brrt wait what 14:45
nwc10 setting build, Stage optimize, nom HEAD
brrt with jit?
or without?
brrt also gets that bug
nwc10 with
brrt oh bloody hell
nwc10 ASAN doesn't spot anything bad 14:46
brrt ugh :-(
timotimo nqp builds in just 30 short seconds 14:49
that's pretty good
jnthn Way better than taking 30 long seconds.
timotimo yep 14:50
jnthn Urgh. If I build CORE.setting with a tiny nursery I get SEGV 14:52
timotimo interestingly, -march=native doesn't change build times
jnthn ah...
void MVM_reentrantmutex_lock(MVMThreadContext *tc, MVMReentrantMutex *rm) { if (MVM_load(&rm->body.holder_id) == tc->thread_id) {
SEGVs there
timotimo: Didn't you add a reentrant mutex recently?
timotimo oh, did the mutex for the compunit not get initialized early enough?
i did, yes 14:53
jnthn Did you GC mark it?
timotimo i think so. gimme a sec.
jnthn Or write-barrier assigning it?
timotimo i gc-mark it
and i assign_ref it
might be wrong
it's in MVMCompunit.c
brrt :-(
jnthn oh... 14:54
MVM_ASSIGN_REF(tc, &(root->header), body->update_mutex, REPR(mutextype)->allocate(tc, STABLE(mutextype)));
That isn't too wise
timotimo dang. i don't grok the assign_ref macro at all >_<
jnthn Well, it's that body can get invalidates by the allocation
Ended up looking at this 'cus i got a not-very-reproducable explosion while building CORE.setting during an earlier change 14:57
timotimo i'll ask for double-checks whenever i use the assign ref macro in the future
jnthn runs it again to see if we look better this time
brrt things where going too good today :-( 15:00
timotimo for "say 1" i'm down to 105 mb maxrss
jnthn Is rss the one including mmap'd stuff? 15:01
timotimo that's a good question
but it used to be 135mb
jnthn Well, on Windows I get around 89mb, is all 15:02
timotimo mhm
could well be that it counts that on linux but not on windows 15:03
brrt hopes something is wrong with newlexotic 15:06
dalek arVM: 43991af | jnthn++ | src/6model/reprs/MVMCompUnit.c:
Correct comp unit mutex setup.
timotimo mhm mhm 15:07
16:20 brrt joined 16:21 brrt left, brrt joined 16:25 flussence joined
brrt uhm, it's kind of difficult to repeat this bug of ours 16:30
timotimo that sounds good. we don't want to repeat bugs, right? :P 16:31
brrt hmmm 16:34
yes and no
we want any bugs that do exist to be repeatable 16:35
timotimo reproducible is the word we're looking for, eh?
brrt yes 16:36
:-)
dalek arVM/moar-jit: 91b60e5 | (Bart Wiegmans)++ | src/jit/graph.c:
Add binddynlex support.

Note that this seems to work, but I've seen spurious, difficult-to-repeat crashes in stage optimize, and nwc10++ has seen them as well.
16:40
brrt any help in reproducing the bugs would be greatly appreciated 16:42
FROGGS ohh, two witnesses O.o 16:43
brrt can you repeat it? 16:44
FROGGS not tried yet... that would happen in rakudo's stage optimize?
timotimo there seems to be a merge conflict when pulling master into moar-jit 16:45
but we probably should get that segfault fix jnthn made above (for the compunit mutex initialisation)
brrt yes
i'll merge master
timotimo oh 16:46
that's not really a conflict
that's two things being added at the same place
brrt :-)
easy enough to resolve i think
hmm 16:47
or not
timotimo it's just that a } got disappeared from the "conflict"
brrt oh i see
dalek arVM/moar-jit: 5494549 | jnthn++ | src/ (5 files):
Make static lexical deserialization lazier.

Only deserialize on first usage of a particular lexical now. Cache it so future lookups are fast. Much work in this patch by timotimo++. ef9e381 | (Tobias Leich)++ | build/Makefile.in: fix build of minilua by moving LDLIBS to the end
  See stackoverflow.com/questions/9253200
16:48
brrt merged :-)
timotimo good :)
FROGGS builds 16:49
16:49 dalek joined
brrt builds too 16:49
timotimo hmm, jit makes the nqp build take a bit longer 16:51
brrt yes, that's what i find as well
bloody! 16:52
anyway, the crash report is clear enough
i'll look at it after dinner
brrt dinner &
16:53 brrt left
timotimo i can get it to be a tiny bit less slow by using -j2 16:53
i get that same error brrt got 16:54
with spesh disabled, it works again 16:56
jnthn If you disable spesh, it won't JIT... :) 16:57
timotimo it breaks when having inline disabled 16:58
FROGGS /home/froggs/dev/MoarVM/3rdparty/dynasm/minilua.c:3496: undefined reference to `floor' 17:01
/home/froggs/dev/MoarVM/3rdparty/dynasm/minilua.c:3497: undefined reference to `pow'
/tmp/ccuGAzX8.o: In function `luaV_execute':
:o(
that's the cmd btw: gcc -O1 -DNDEBUG -g3 -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -DMVM_TRACING=0 -DMVM_CGOTO=1 -lm -lpthread -lrt -ldl 3rdparty/dynasm/minilua.c -o 3rdparty/dynasm/minilua 17:03
/tmp/ccuGAzX8.o: In function `constfolding':
timotimo i thought -lm had that? 17:20
nwc10 m: say 5.6892e+01/5.7454e+01; say 5.6892e+01/6.597e+01 17:29
camelia rakudo-moar 5d5ec1: OUTPUT«0.990218261565774␤0.862391996361983␤»
nwc10 Stage optimize : Cannot look up attributes in a type object at src/Perl6/Optimizer.nqp:606 (blib/Perl6/Optimizer.moarvm:lexical_vars_to_locals:4294967295) 17:40
that's an interesting line number
moar-jit HEAD, master, nom 17:41
moar's master is fine
timotimo *sigh*
FROGGS I am a JIT hacker \o/
timotimo yeah, linking order
FROGGS Stage optimize : Cannot look up attributes in a type object 17:52
at src/Perl6/Optimizer.nqp:606 (blib/Perl6/Optimizer.moarvm:lexical_vars_to_locals:4294967295)
from src/Perl6/Optimizer.nqp:867 (blib/Perl6/Optimizer.moarvm:visit_block:0)
nwc10 same as mine. With the same interesting line number 17:53
timotimo how come we generate so many gotos directly before the label they point to?
FROGGS O.o
that does not make much sense 17:54
timotimo i mean
only one goto each
but we do that often
FROGGS 606 is the line after the comment: 17:55
# Also ensure not dynamic.
my $dynamic := try nqp::getattr($qast.value, nqp::p6var($qast.value).WHAT, '$!descriptor').dynamic;
so $qast.value is probably NQPMu 17:56
timotimo gist.github.com/timo/b2420e6f5db8d5791266
just from looking at optimizer.nqp.moarvm a bit
FROGGS ohh dear 18:00
it is a Heisenbug like the NFA thing...
timotimo noooo :< 18:01
FROGGS when I do the following before line 606 it succeeds:
nqp::say($qast.value.HOW.name($qast.value));
jnthn If it looks like a try not working, or working flakily, given handlers were recently enabled in the JIT, that could be a hint... 18:03
FROGGS now I get: 18:06
Stage optimize : Spesh: failed to fix up handlers (-1, 672, 672)
at <unknown>:1 (blib/Perl6/Optimizer.moarvm:lexical_vars_to_locals:4294967295)
(I added this before line 606: unless nqp::istype($qast.value, NQPMu) { )
might be easier to debug, I dunno
timotimo jnthn: superfluous gotos are not terribly expensive, aye? they don't cost much to interpret and having more BB's in spesh won't be the end of the world either, right? 18:09
how costly is it to const_s the very same string into registers multiple times over?
jnthn That's cheap, and probably JITs really cheap 18:10
timotimo the most costly part is probably causing speshing and jitting to do more work unnecessarily 18:11
jnthn The surplus gotos aren't a huge deal, but would be nice to dispose of
dalek arVM: ec09497 | jnthn++ | src/core/ (5 files):
Fix --dump when there are state variables.
18:13
18:33 brrt joined
FROGGS damn, my pseudofix for the NFA bug does not work anymore :o( 18:34
brrt rather hopes this is a bug in handlers 18:39
18:48 woolfy joined
jnthn Today's fun story: earlier on this evening I left a spectest run to happen while I went to cook dinner. Upon returning to my computer after dinner, it was utterly hosed: loudly swapping, and unresponsive to pretty much anything. 18:56
brrt hmm, disabling inlines or disabling OSR doesn't change it
ok, /me wonders how this will end
jnthn A restart later, I watched it spectest. And found I'd broken slurp.t in such a way that one test looped and allocated at a rate of about 500MB/s :) 18:57
In other words, 30s to fill all available RAM :)
brrt uhm... 30s * 500MB/s is .... 15GB ? 18:58
not bad
jnthn Yeah, I got 16GB of RAM in here
I don't know how far it made it into the swap :)
Anyway, turns out it was the same bug that had broken encode.t the other day :) 18:59
FROGGS jnthn: you are working on string ops optimization?
jnthn No!
Was improving code-gen believe it or not :) 19:00
FROGGS ahh :o)
jnthn I got rid of a ton of useless wval and getlex operations we were code-gening in the mainline.
brrt gcc without optimize is truly ridiculously fast