00:28 woosley joined 01:22 FROGGS_ joined 01:29 woosley left, woosley joined 05:38 brrt joined
brrt .tell timotimo that only a really simple frame would be compiled, but spesh /should/ try to compile a frame that is invoked enough times 05:39
that compiled function might well be wrong, i.e. there may be things that i've overseen
timotimo oh, neato 05:43
will your next blog post have examples of super simple frames that can already be compiled and maybe the resulting assembly? 05:44
brrt i hope so, yes 05:46
i'm preparing how to make that super simple compiled frame :-) using MAST directly 05:47
timotimo i recall that part
sounds good :)
am i understanding correctly that you're basically taking ops that are little more than 1) get arguments, 2) invoke a C function, 3) set result register? 05:48
and turning them into direct access to the work space and an invoke directly to the C function?
brrt yes, its a little more than that already, but that is the gist of it 05:51
calling a c function was quite a big deal so i worked quite a bit on it
but i'm off for now, have to get bread for breakfast
timotimo OK :) 05:52
06:39 brrt joined 06:46 zakharyas joined 07:01 FROGGS joined 07:09 oetiker joined 07:24 oetiker_ joined
nwc10 jnthn: (Un)success - ba6b4e37f8d99 breaks NQP -- backtrace from valgrind: paste.scsys.co.uk/392691 07:35
ASAN colo(u)rfully unhappy too 07:36
07:36 oetiker joined
nwc10 jnthn: OK, that ASAN explosion is (somewhat) late. Previous MoarVM on origin/inline will spectest under ASAN, but a regular debugging build with SEGV in the NQP build 07:42
the cause of the SEGV appears to be strongly related to the *first* error in the valgrind report:
==26472== Conditional jump or move depends on uninitialised value(s)
==26472== at 0x4FE6363: build_cfg (graph.c:325)
so might be 2 bugs, or the ASAN failure might be the first bug finally causing an unhappy code path to be taken
jnthn nwc10: And this exists on HEAD also? 08:06
The 325 one is odd... 08:07
for (i = 0; i < g->num_inlines; i++) {
The spesh graph is calloc'd 08:08
(that's what's in g) 08:09
Ah... 08:10
nwc10 the undefined behaviour appears to be introduced by 54d3380fe92910f1845b358219dc5f2ba03d4137
Start building inline extents table.
clean build with 42fee8647314e83a326484a7b5d43fb478bf412d
jnthn I think the value ->num_inlines was initialized with was uninitialized 08:11
nwc10 now running valgrind having found a clean build
have run 3 previously failing build commands with valgrind at 42fee8647314e83a326484a7b5d43fb478bf412d 08:16
all happy.
dalek arVM/inline: 2c81704 | jnthn++ | src/spesh/candidate.c:
Ensure we don't read junk memory in a candidate.

  nwc10++ for reporting the Valgrind/ASAN error that was probably caused
by this.
nwc10 so I think that 54d3380fe92910f1845b358219dc5f2ba03d4137 is the culprit for the undefined behaviour 08:17
OK, I've cherry picked that one onto 54d3380fe92910f1845b3 and it hasn't blown up yet 08:18
jnthn ok
bbi10
nwc10 will run that to completion, then try (new) origin/inline HEAD
jnthn thanks! 08:19
nwc10 take your time - I think it will take about 20 minutes
FROGGS nwc10++
08:34 brrt joined
jnthn back 08:38
morning, brrt
brrt hey, jnthn
i've heard from mike pall wrt dynamic registers, i'll forward it to you
jnthn k
brrt how's things :-) 08:41
jnthn brrt: That's a helpful response. :) 08:47
brrt indeed it is :-)
jnthn And means that dynasm isn't going to be a dead end. :)
FROGGS can I see it?
jnthn (e.g. we can go further with it once we've got the simple thing working)
brrt: Things are fine, just doing the day's first coffee, done email reading, about to get back to inlining :) 08:48
brrt oh, yes, i'll gist it
FROGGS: gist.github.com/bdw/e4ed70c4bd0de4f5d276 08:51
FROGGS thank you
brrt yes, i'm really happy to hear this
FROGGS very very nice 08:53
moritz doens't understand much of that
brrt moritz: context is extending DynASM to use dynamic registers on x86_64 and ARM 08:54
x86_64 being the main target, ARM the second
they already work on x86, and I had assumed they worked everywhere
timotimo that sounds great! 08:55
moritz and we need dynamic registers, because otherwise we'd have to implement a register allocator in moarvm?
brrt no, just the other way arround :-)
basically, if i want to compile: a = b + c, then thats a load b, load c, add b, c, store a, b , right? 08:56
but, i need to know where a and b are loaded to emit the add instruction 08:57
moritz ah
brrt a 'real' compiler does this recursively
or dynamic-programmingly, not important for this discusion :-) 08:58
but that means you have to emit an instruction where you fill in the exact register later
i.e. at runtime
dynasm could do this but only for x86 08:59
moritz ok, now I understand
thakns
brrt :-)
for complete understanding, note that the alternative is to have the registers for the load's fixed (static allocation)
moritz I guess the alternative would be use a fixed set or registers, and push the old values on the stack, and restore them later 09:00
brrt exactly
moritz but we want to avoid that :-)
brrt more importantly - imho - if you do that, there is nothing intelligent you can do with regards to load / stores
i.e. you can't say 'this value will be used 3 instructions later, and that will be the live value on the end of the basic block, so i can skip the store and emit it after the next instruction using it' 09:01
that is not so important for now, as the first goal is to get jit compilation working at all
(according to #luajit, this is all made significantly more complex by the fact that modern cpu's actually rename registes and apparantly can keep top stack values in registers, but we'll not worry about that soon) 09:03
timotimo yeah, in modern cpus, what your actual assembly instructions do and what the cpu does (and in what order) is ... different :) 09:10
nwc10 jnthn: the other 2 errors reported by valgrind in the paste remain, and are independent of the bug that you found and fixed 09:16
and cause ASAN to be very unhappy and vomit ANSI codes. 09:17
jnthn Do you have the output against HEAD with latest line numbers?
nwc10 other 2 errors were introduced by ba6b4e37f8d99a9eaf3d20152052aa6e1024a153
coming soon... 09:18
jnthn: paste.scsys.co.uk/392732 09:20
do you need longer stack backtraces? 09:21
jnthn No, I'll try and work from this; thanks. 09:22
4 bytes before is significant, 'cus -1 gets used as a "no deopt entry here" sentinel value 09:23
*but* I'm very curious why that patch introduced it where it wasn't there before.
dalek arVM/inline: c00f594 | jnthn++ | src/spesh/inline.c:
Start detecting coderef/str/callsite refs to fix.

They need fixing up in the case some some inter-compilation-unit inlines. For now, just complains (so we know that's actually the problem).		 09:24
arVM/inline: 0cfd4af | jnthn++ | src/core/frame.c:
Ensure args always go in the right place.

That is, we should always have | registers | args |. We got this wrong in the inline case until now, overlaying the args buffer with some of the inlinee's registers.
jnthn (no, those aren't about it)
nwc10 MOAR backtrace is a command line option, so easy to change: paste.scsys.co.uk/392735 09:25
jnthn: even more current line numbers, after the previous 2 09:29
paste.scsys.co.uk/392738
AFK&
dalek arVM/moar-jit: 9848c72 | (Bart Wiegmans)++ | src/ (4 files):
Indentation / whitespace fixes
 09:54
arVM/moar-jit: a4fa5f9 | (Bart Wiegmans)++ | src/ (6 files):
Rewritten JIT graph construction to do most of the work.

This effectively moves most of the logic out of x86_64.dasc and into the jit.c file.
brrt &lunch
dalek arVM/inline: a0db232 | jnthn++ | src/ (4 files):
Prepare for comp units gaining extra constants.

This is how we'll handle str/callsite/coderef constants when we inline code from another compilation unit.		 10:33
arVM/inline: ea9d487 | jnthn++ | src/ (3 files):
Handle cross-cu callsites in inline.
arVM/inline: d0466b6 | jnthn++ | src/spesh/inline.c:
Fix cross-cu wvals in inline.
10:36 brrt joined
dalek arVM/inline: a1ad3b2 | jnthn++ | src/spesh/inline.c:
Fix a thinko.

Just 'cus the code look like it'll do the right thing doesn't mean it actually will...		 10:47
arVM/inline: c079349 | jnthn++ | src/spesh/graph.c:
Add missing sanity check in deopt rebuild.

Hopefully resolves issue valgrind picked up reported by nwc10++.
nwc10 was actually ASAN. I just used valgrind to get the stack traces 10:49
jnthn ASAN++ 10:52
10:54 woolfy1 left
FROGGS from reading the commits it seems like inline is not trivial :o) 11:01
ohh, and with Python a1ad3b2e85 would not be needed 11:03
dalek arVM/inline: bbcbdec | jnthn++ | src/ (3 files):
Fix up cross-CU string constant usages.
 11:11
arVM/inline: 20cc072 | jnthn++ | src/spesh/inline.c:
Fix operand thinko in wval fixup.
nwc10 jnthn: our old friend t/spec/S17-lowlevel/lock.rakudo.moar still fails 11:12
t/spec/S17-supply/batch.t failed during the test, but passed on a re-run
otherwise, all happy
so yes, you seem to have found and fixed the bug
jnthn :P 11:14
FROGGS: No, it's not trivial. Trickiest thing I've worked on for a while.
11:15 Util joined, woolfy joined, woolfy left
jnthn nwc10: Nice :) 11:23
nwc10 jnthn: after your latest changes, all is fine apart from the perennial t/spec/S17-lowlevel/lock.rakudo.moar 11:38
what's the overhead of making a call? Perl 5 has no resource allocation in the steady state, so is MoarVM similar? 11:43
jnthn "It depends" 11:44
But typically calls should hit the frame cache, which avoids allocating.
nwc10 that's good enough for my arm-wavy "steady state" 11:45
brrt jnthn, if i have a bytecode segment constructed in nqp, compiled with the MAST compiler, how do i invoke it? 11:47
11:47 FROGGS joined
brrt or, what kind of object does the mast compiler return, anyway 11:48
nm, i'll get there eventuall 11:49
eventually :-)
jnthn A compilation unit 11:55
Try nqp::compunitmainline($the_thing)
Which should give you an invokable thingy.
If all else fails, just gist me what you have and I'll take a look :)
brrt ok, i'll try :-)
where are the nqp:: ops defined, anyway
jnthn src/vm/moar/QAST/QASTOperationsMAST.nqp 11:58
brrt add_core_moarop_mapping 12:00
is see
*i 12:01
it works, by the way 12:03
jnthn "it"? :) 12:05
moritz the JIT compiler? :-)
brrt no, not yet that, the compunitmainline thingy 12:06
:-)
i expect big firework crashes on the jit compilation
is there a way to get something like 1..10 in nqp? 12:07
moritz I don't think so 12:08
brrt it doesn't matter
brrt and his hll conveniences 12:09
12:13 lue joined 12:14 jnthn joined
brrt ugh nearly there... 12:21
.... const_i64_16 12:38
fun!
yes, sigsegv! 12:42
12:42 colomon joined
jnthn You and me both... :( 12:44
timotimo has frozen joghurt instead of segv 12:47
12:48 _sri joined
nwc10 jnthn: have you considered buiding your tree on your Linux VM and seeing what valgrind says about the SEGV? 12:49
jnthn nwc10: The debugger tells me that it's an object with a corrupt STable point 12:50
*pointer
nwc10 bother
timotimo did you know you can get a gdb backtrace from an asan error?
nwc10 knows that, and sometimes can remember the thing to stick a breakpoint on 12:51
but if it's a heap error, I prefer to get valgrind to do it
jnthn oh...I think I might now what it is
timotimo OK
nwc10 as valgrind will also give me uninitialised variable errors, which might be the real cause of the bug
timotimo jnthn: you might now?
jnthn yeah, fail.
*know
timotimo that sounds good :)
12:52 FROGGS joined
jnthn yeah. darn. 12:52
ugh.
brrt is there a way to compile moar with debug symbols? 12:54
jnthn Just configure it with --debug
brrt thanks :-) 12:55
12:56 krunen joined 12:58 _sri joined 13:02 colomon joined
brrt clearly doesn't know all there is to now about dynasm labels 13:03
timotimo much now today 13:04
brrt hmm somehow i don't think that 8 is a proper value for a pointer-to-mvmthreadcontext 13:07
moritz small pointer values are moar human-friendly; they allow you to build a personal relationship with the pointer! 13:09
brrt moar is nothing if not human friendly 13:10
why you no dwim 13:11
13:13 ashleydev joined
brrt ahum.... case: break 13:13
corrupt bytecode stream oh goodie 13:14
jnthn Maybe the return went to the wrong place? 13:17
dalek arVM/inline: 1779732 | jnthn++ | src/spesh/dump.c:
Teach the spesh dumper about sslot operands.
 13:19
arVM/inline: 7e4b6b6 | jnthn++ | src/spesh/inline.c:
Correct re-writing of remote-cu wvals.

We can't just do this as we go, otherwise we end up with the rewrites of the spesh slots pointing to wrong things.
jnthn Yay, no SEGVs in make test of NQP now. Just a bunch of failures...
13:19 _sri joined
jnthn Don't know how to deopt_all from inline yet 13:20
And that one is just an NYI :)
brrt looks like it (wrt return)
although, that'd be weird 13:21
jnthn makes a cup of tea before attacking deopt_all 13:28
The other bug I notice is something to do with lexicals...hmm.
dalek arVM/moar-jit: bde5170 | (Bart Wiegmans)++ | foo.nqp:
Added a test for JIT compilation.
 13:29
brrt has a sneaking suspsicion something is going wrong in pointer-manipulating dasm 13:42
on the other hand, evidence of that is not so good 13:43
13:45 _sri joined
brrt is off for a bit 14:06
14:06 brrt left
dalek arVM/inline: 583c69e | jnthn++ | src/ (5 files):
Restore ->code_ref and ->outer in deopt frames.
 14:09
arVM/inline: f5a4b36 | jnthn++ | src/spesh/deopt.c:
deopt_all of frame with inlines: main body case.

This makes deopt_all work when we encounter a frame with inlines to deopt, but we're not currently in an inlined piece of code, so there are no frames to re-create.
timotimo there can't be much left now :) 14:10
FROGGS timotimo: I also look at the branch name first, and hope it already hit master :o) 14:19
jnthn 16 NQP tests still fail 14:21
Didn't even try an NQP build.
FROGGS rakudo build you mean? 14:39
jnthn No, NQP 14:40
FROGGS how do you... ahh! 14:41
:o)
jnthn I just have one already built :) 14:42
nwc10 jnthn: current stuff is into a setting compilation
FROGGS yeah, I was thinking that one has to build nqp in order to test it... but that is soo old fashioned :o)
nwc10 going home, may not be able to tell you whether it passed spectest for a while
OK, it's got past the Rakudo tests and is into spectest 14:43
timotimo is off for a brrt
14:59 colomon joined
dalek arVM/inline: c46aab5 | jnthn++ | src/spesh/deopt.c:
First crack at deopt_all uninlining.

Seems to work - or at least, makes 4 more tests pass. One test that failed on deopt_all not being implemented does still fail, but that may be unrelated.		 15:31
arVM/inline: 3b65cd7 | jnthn++ | src/core/ (2 files):
Avoid looking at junk memory in arg context.

A deopted frame doesn't have arg processing set up.
nwc10 jnthn: the usual, plus t/spec/S32-io/IO-Socket-Async.t failed, passes on a re-run 15:34
jnthn k 15:38
dalek arVM/moar-jit: ae6e4ed | (Bart Wiegmans)++ | src/ (6 files):
Invoke a JIT function. Returning doesn't work yet.
 15:54
arVM/inline: 9baf6a6 | jnthn++ | src/spesh/codegen.c:
Sanity check to ensure inline offsets are written.
 15:57
arVM/inline: e9c55d9 | jnthn++ | src/spesh/manipulate.c:
Move inline start annotations when deleting ins.
jnthn Turns out that last one accounted for all manner of bother.
Not sure I quite believe it, but I'm down to 1 failing NQP test now.
Spesh inline: unexpected num_pred 15:58
hmm
FROGGS O.o
jnthn That sounds like "the impossible happened"... :)
FROGGS jnthn++
dalek arVM/moar-jit: b7f0f73 | (Bart Wiegmans)++ | src/core/interp.c:
Returning from JIT now works.

It turns out that when you have to return from your frame, you really have to return from your frame, and not just forget it and hope for the best :-)		 16:08
jnthn Hmm, in the long run we'll want a variant of try return that is suitable for the JIT to caller. But yeah, that works for now :) 16:11
dalek arVM/inline: cf95c3e | jnthn++ | src/spesh/inline.c:
Liberalize pred tweaking during inline.

We sometimes get a topology where the invoke is already the final thing in a basic block, and other things jump to immediately after it (mostly clearly happens with invoke_v). Cope with this scenario.		 16:22
jnthn Well, that's NQP tests clean with inline enabled. 16:23
Amazignly, that's with GC updates not done.
Rakudo's build blows up at Grammar.nqp, which I'm hoping is thanks to that. Cool it gets through the steps up to that, mind... 16:24
OK, walk, dinner, GC updates to cope with frames carrying inlinings, then will see if I can get NQP building atop of moar with inlining. 16:26
17:06 FROGGS joined
FROGGS brrt++ # b7f0f73 :o) 17:07
17:35 lizmat joined, vendethiel joined
nwc10 jnthn: nothing broken at cf95c3eb8ed0221 17:38
17:38 woolfy joined 17:41 brrt joined 17:43 woolfy joined 17:53 lizmat joined 18:00 woolfy joined
nebuchadnezzar o/ 18:15
FROGGS hi nebuchadnezzar
nebuchadnezzar is /usr/lib/MAST required in that place? couldn't it be installed under /usr/share/ since the .nqp files seems arch independent? 18:19
it seems to be “planned” in FROGGS comment for NQP issue github.com/perl6/nqp/issues/154#is...t-33594960 18:20
FROGGS yes, but I had not enough time to do that actually :/ 18:22
.nqp files are source code files, like .pl
nebuchadnezzar I may do it as I need it to respect Debian policy ;-) 18:23
FROGGS that would be awesome :o)
nebuchadnezzar so I move it and modify nqp tools/build/Makefile-Moar.in to use the new path 18:24
I wonder if nqp shouldn't use some pkg-config like stuffs to find where moar stuffs are 18:25
FROGGS that seems to be a good long time goal 18:26
nebuchadnezzar ok, will fill a request after diner ;-) 18:28
18:36 brrt joined
brrt somethings odd with the return value 18:43
jnthn Try adding two even numbers, or two odd numbers? :P 18:46
brrt well, it increases with the run loop count 18:47
so it seems somehow the jit frame is looking at the work registers of the parent frame
jnthn o.O
brrt yes 18:48
jnthn tssk, is github down...
dalek arVM/inline: 7d5d5d6 | jnthn++ | src/gc/roots.c:
Take into account inlining when GC marking frames.
 18:49
jnthn ah, worked second attempt
Didn't try the NQP build yet, but the Rakudo one with inlining enabled now makes it to CORE.setting before exploding.
(And explodes 'cus a sanity check fails. So it knows its wrong.)
brrt hmm... it seems i have to propagate the MVMCompUnit through the spesh graph to get it to the jit compiler 19:08
of
or
or not... hmm
jnthn Comp unit? 19:09
That's reachable through various means
including tc->cur_frame->static_info->body.cu or so
brrt that may be better 19:10
nwc10 jnthn: fine at 7d5d5d6 19:11
19:33 raiph joined
masak nwc10: what is it you're testing, exactly? 19:33
jnthn masak: Making sure my inline branch - without the inlining enabled yet - doesn't bust the NQP/Rakudo build or introduced any errors asan can find. 19:41
masak ah. 19:42
nwc10++
jnthn: but you're building NQP/Rakudo with the inlining enabled, yes? 19:43
jnthn No, 'cus up until an hour or two ago it didn't even pass the NQP tests.
masak oh! 19:46
dalek arVM/inline: 71f3cde | jnthn++ | src/core/frame.c:
Fully NULL arg-related bits in deopt frames.
 19:47
arVM/inline: 01efeea | jnthn++ | src/spesh/ (3 files):
Never delete inlined BBs.

Technically, they may correctly become unreachable. For now, though, we rely on the inline annotations not vanishing.
arVM/inline: 9fa0cf3 | jnthn++ | src/spesh/manipulate.c:
Make annotation movement a little more robust.
jnthn With those it gets a good way into the setting of Rakudo 19:49
I'm currently trying an NQP build.
Much wow, it completed. 19:50
And the resulting NQP passes all the NQP tests.
nwc10 pinch yourself? 19:51
jnthn :P 19:52
Well, will still need to work out why the Rakudo setting build explodes.
Plus do an optimized build with spesh logging turned off to make sure it's actually faster... :) 19:53
oh, no explode this time 19:54
Error while compiling op p6decontrv: Spesh: failed to resolve extop in code-gen 19:55
ohhh
extops
hm.
dalek arVM/inline: d187eb9 | jnthn++ | src/spesh/inline.c:
For now, don't try to inline anything with extops.
 20:02
timotimo is getting excite
raiph d
[Coke] ment 20:03
jnthn Still a SEGV, sadly. 20:05
[Coke] exciteSEGV? 20:13
brrt back at segv :-) 20:26
20:29 bcode joined
jnthn ah, phew, finally it segv'd under the debugger at a point that provided a godo clue. 20:33
*good
nwc10 jnthn: still good at d187eb93f5a381c67e9822837e2451db87ba3688 20:34
brrt ugh, callee save registers 20:36
dalek arVM/inline: 2a7aa82 | jnthn++ | src/spesh/inline.c:
Remember to initialize lexicals_start.

Otherwise deopt will go copying all kinds.		 20:39
20:39 masak joined, sergot joined 20:41 camelia joined
jnthn Seems it survives CORE.setting now, including with an optimzied build 20:49
dalek arVM/moar-jit: 75a683a | (Bart Wiegmans)++ | / (6 files):
Almost fixed returning a value.

Bug is that whatever I do, the routine returns 81 rather than 2 as I'd expect. It does somehow return the right value from the result register.
Also added preliminary support for say() and const_s, but I need to figure out whether to load the pointer at runtime or at compile time, in which case I'll need to find the CompUnit then.		 20:51
timotimo like teh sound of those :) 20:56
nebuchadnezzar \o/ beer powered 20:58
jnthn So, remaining task is to re-do the whole frame memory caching thing. 21:27
'cus now any frame with something inlined in it just goes of to malloc, due to the variable sized thing. 21:28
Turns out that this is sufficiently inefficient it basically eats the benefits of inlining, or so it seems.
dalek arVM/inline: 76c98cc | jnthn++ | src/spesh/optimize.c:
Turn on inlining.

Note that due to various restrictions, there are many things it will not inline yet. Additionally, all spesh'd frames of non-standard size fall back to malloc'ing the frames, which is enough of a pig to eat many of the gains we get so far.		 21:32
timotimo the pig eats all the grains? 21:46
jnthn Pretty much, though I knew this would need dealing with. 21:47
raiph Would redoing the frame mem stuff be an LHF for a decent C hacker? 21:49
jnthn: I would be happy to post links to another of your awesome blog posts about progress, especially one that mentioned a couple LHFs with likely nice speedup payoffs. (Though, knowing you, by the time you read this you'll have not only worked out how best to redo it but done the coding too.) 21:55
jnthn Well, I'm on with the coding, but might leave it until tomorrow to finish it up :) 21:57
Granted the lock-free algo it wants is easy, but still...it's almost midnight :)
timotimo "malloc'ing the frames" means bypassing the frame cache? 21:59
jnthn timotimo: yeah 22:00
timotimo mhm, that does sound somewhat costly 22:01
will inlining be dumped in the regular spesh log? 22:02
22:03 japhb_ joined
jnthn Well, you can see where it happened 22:03
Just search for Inline Start in the log 22:04
timotimo oke 22:15
jnthn Anyway, I think I probably hacked enough stuff up today :) 22:16
Got some fun (really) $dayjob thing to hack on tomorrow, but will work on the allocator thingy in the evening.
timotimo :)
22:30 lizmat joined, woolfy joined
jnthn 'night o/ 22:40
lizmat gnight jnthn!
22:49 vendethiel joined 23:12 woolfy left 23:46 cognominal joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!