github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
jnthn That's to be continued tomorrow :) 00:01
Though I think I'm much of the way through it
Geth MoarVM: d0be1847a1 | (Daniel Green)++ | 2 files
Fix typos in comments
 00:17
MasterDuke those new commit bits go stale quickly if you don't jump on them... 00:19
jnthn \o/
01:37 MasterDuke left 03:11 MasterDuke joined 03:15 MasterDuke_ joined 03:18 MasterDuke left, MasterDuke_ is now known as MasterDuke 04:23 Kaiepi joined 04:40 brrt joined
brrt \o 04:42
05:09 btyler_ joined 05:11 avarab joined, avarab left, avarab joined 05:16 buggable left, SmokeMachine left, chansen_ left, huggable left 05:35 buggable joined, SmokeMachine joined, chansen_ joined, huggable joined 06:16 bisectable6 joined, greppable6 joined, nativecallable6 joined, statisfiable6 joined
samcv we can save another 4 bytes per hash handle if we pack the struct 06:38
Geth MoarVM: fae91a6ebf | (Samantha McVey)++ | src/strings/uthash.h
Fix indentation of code in uthash.h

Align everything to four spaces of alignment.		 06:41
06:50 robertle joined 06:52 domidumont joined
samcv though it does seem to make it slightly slower 06:55
06:58 domidumont left, domidumont joined 07:29 brrt left 07:48 domidumont left 07:49 domidumont joined 07:57 zakharyas joined 08:06 brrt joined 08:08 zakharyas left 08:31 zakharyas joined 08:50 domidumont left, domidumont joined 08:51 domidumont left 08:52 domidumont joined 08:58 brrt left
jnthn morning o/ 09:03
09:07 dogbert2 joined 09:11 brrt joined 09:16 domidumont left 09:17 domidumont joined 09:27 brrt left 09:28 lizmat joined 09:31 domidumont left 09:32 domidumont joined
samcv good * 09:43
jnthn o/ samcv 09:45
Geth MoarVM/ctx-lazy: aae518ba0b | (Jonathan Worthington)++ | 7 files
Update lexprimspec to do context traversal
 09:47
MoarVM/ctx-lazy: 2194358c09 | (Jonathan Worthington)++ | src/6model/reprs/MVMContext.c
Die more politely in MVMContext clone
 09:54
MoarVM/ctx-lazy: 50262e3ca4 | (Jonathan Worthington)++ | src/6model/reprs/MVMContext.c
Update bind_key in MVMContext for traversal
10:04 zakharyas left
Geth MoarVM/ctx-lazy: 5b0014f284 | (Jonathan Worthington)++ | 5 files
Update ctxcode to do context traversal
 10:05
10:05 zakharyas joined
dogbert2 jnthn: is you current work an attempt to fix the SEGV's in some stresstest or is it some kind of opt? 10:10
jnthn dogbert2: It's trying to fix context introspection and all the things around that to cope with inlining having taken place
dogbert2: Which will hopefully fix a couple of module regressions 10:11
dogbert2 I was wondering because one of the SEGV's mentions ctx but perhaps that's a coincidence
jnthn Though we actually could have run across this bug before now, it was just less likely when we were a tad less keen on inlining some things
Hm, what SEGV is that? 10:12
dogbert2 t/spec/S17-promise/lock-async-stress.t - 0xb7b22e1c in MVM_interp_run (tc=tc@entry=0xa391648, initial_invoke=initial_invoke@entry=0xb7b451a0 <thread_initial_invoke>, invoke_data=0xb0fb1330) at src/core/interp.c:3933
3933 if (ctx->callsite->num_pos != ctx->callsite->arg_count)
with ctx->callsite being null
jnthn Hm, interseting 10:13
Though that's not directly related to what I'm working on
dogbert2 doesn't happen very often though
jnthn I'm fairly close to having the ctx-lazy branch able to pass the NQP test suite (already bulds it fine) 10:15
*builds
dogbert2 I can try it out on my 32 bit system when you're happy with it 10:17
Geth MoarVM/ctx-lazy: f428010ad0 | (Jonathan Worthington)++ | src/spesh/frame_walker.c
Correct inline handling thinko in move to outer
 10:21
MoarVM/ctx-lazy: afe4fc52c1 | (Jonathan Worthington)++ | 3 files
Update MVMContext elems for traversal
 10:27
jnthn That branch (plus local changes to the ops that set up traversals) now passes the NQP tests 10:28
There's still 3 places I need to update; I figure they aren't important for NQP's tests :)
dogbert2 impressive 10:31
jnthn hm, no, one test fails 10:34
dogbert2 well, soon to be impressive then :)
jnthn ah, nice, two NQP tests fail if I make the things I didn't update yet panic. So I can fix them up in NQP first 10:35
oh goodness, I need to update MVM_frame_find_contextual_by_name 10:37
It should become far simpler, in theory, because it won't need to contain the inlines handling 10:38
lizmat eek :-)
jnthn Which is now factored out to frame_walker
nwc10 samcv: I loose track. Did "we" ever work out why hash values weren't in the bucket that they were supposed to be in? 10:42
samcv nwc10: well they were in the right bucket. it was an issue that they were just keyed differently than i expected 10:57
nwc10 aha OK.
samcv so i fixed the issue and just rely on the hash handle's stored value instead of trying to calculate it myself
nwc10 I'm glad the bug has been identified, and the relevant wetware patched :-) 10:58
samcv so i can go to the corrct bucket and then find it in the chain, then delete it
11:01 buggable left, buggable joined 11:02 robertle left 11:12 zakharyas left
Geth MoarVM/ctx-lazy: 7314e23105 | (Jonathan Worthington)++ | 3 files
Redo dynamic variable lookup using frame walker

So that we don't have the inlines logic (complete with JIT and non-JIT branches) repeated in this code. Also means we get to avoid an amount of repetitive code that was in there.		 11:14
jnthn 3 files changed, 72 insertions(+), 126 deletions(-)
nwc10 negative productivity!
lizmat hehe 11:15
ok, not sure whether this is an artefact in profiling, or a sign that the MoarVM interpreter is blindingly fast if it needs to be
profiling this: my int @a = ^5_000_000
shows only RED bars, indicating that everything is done in the interpreter 11:16
oddly enough, the equivalent code in Perl 5: my @a = 1 .. 4_999_999
is about 2x as slow
jnthn lizmat: Time it with MVM_SPESH_DISABLE=1 11:17
lizmat jnthn: I will
jnthn: without profile or with ?
jnthn Without 11:18
lizmat with MVM_SPESH_DISABLE=1 it's actually faster still 11:19
jnthn huh, so it really isn't doing anything with it
I'd have expected it to hit OSR or something
lizmat yeah, me too 11:20
jnthn If it's really not being JITted then we can expect to win by more once it is :)
lizmat indeed :-)
jnthn OK, lunch, and then will try to finish up ctx-lazy :) 11:21
11:56 robertle joined
timotimo i have made a speshlog of that code and i do see STORE(:initialize) from native_array being speshed and jitted "successfully" 12:18
lizmat ok, so it is a --profile artefact that we don't see it in there
timotimo not quite
i also see the OSR hits go up and up and up after the specialization 12:19
in other words: the specialization gets made, but the code keeps running the non-speshed code
lizmat ah?
jnthn Interesting :)
timotimo it could be the same situation that i found earlier where there's a mismatch between the callsite it expects (so it grabs the OSR result) and the callsite it stores
i.e. it keeps logging hits for one callsite, but when polling for a result it provides a callsite that nothing has been made for 12:20
jnthn wonders how that happens 12:21
lizmat should I ticket it ?
jnthn Sure
I've been pondering that we store OSR candidates separately from the others 12:22
lizmat moar queue, I guess
jnthn And store them by OSR index
And we turn osrpoint instructions that we didn't see any hits for yet into a deopt back to the interpreter
Which will then gather data on the stuff around that point too
Thus letting us deal with the two consecutive loops being poorly optimized 12:23
timotimo ah, yes 12:24
12:25 brrt joined
brrt \o 12:25
timotimo o/
brrt ohai timotimo 12:26
i'm finally seeing the bright side of my proposed expression tree refactor
lizmat M#909 12:27
synopsebot M#909 [open]: github.com/MoarVM/MoarVM/issues/909 Simple benchmark does not show up in --profile as being speshed / JITted / OSRed
12:28 zakharyas joined
brrt thank you lizmat 12:29
12:30 MasterDuke left
jnthn grr, I suspect my re-working of the contextual lookup may turn out to have broken something if the top frame has inlines 12:38
brrt not an easy thing to refactor
still
brrt is very happy jnthn++ has tried 12:39
jnthn Well, the frame walker needs to learn about cases where the top frame on the stack has inlines, is all
Which it didn't have to so far
brrt top frame is frame with NULL caller?
jnthn No, as in the one currently executing 12:40
12:40 domidumont left
brrt ah, i see 12:40
i'm thinking of the stack in the wrong order
(must be that x86 creeping up on me)
12:40 domidumont joined
jnthn
.oO( Typical assembly programmer, thinking stacks always grow down :P )
 12:40
brrt i'm having infinite problems with java's ArrayDeque 12:41
as in, double-ended queue, what side is what?
I haven't looked at all that is in this branch, but do you plan to implement this for e.g. throwpayloadcallerlex and friends? 12:42
jnthn Just for symbol lookups and everything around MVMContext for the branch 12:43
After the branch, may make backtraces use it
So that we don't get lost backtrace entries
brrt nice. I think the frame handler search can also probably benefit 12:45
'search_frame_handler_lex' is what i'm thinking off
jnthn Maybe so, yes
brrt well, I suspect that code to be wrong (or incomplete) in ways relevant to JITting 12:46
as in, i've tried to implement throwpayloadcaller and related a few times and failed, and I've never been able to demonstrate that this was the root cause, but it always looked suspicious 12:49
jnthn oh 12:58
getdynlex and binddynlex are already marked :noinline
So I didn't break them :)
Though this does mean we'll have a path to being able to inline them in the future. 12:59
Geth MoarVM/ctx-lazy: 95d5ac4b78 | (Jonathan Worthington)++ | 5 files
Update getlexreldyn for context traversal
 13:02
jnthn I still wonder quite how we got away with doing these changes for so long :) 13:03
13:05 zakharyas left, zakharyas joined
Geth MoarVM/ctx-lazy: 45bb7016ed | (Jonathan Worthington)++ | 5 files
Make getlexrelcaller traverse the MVMContext
 13:13
13:14 domidumont left, domidumont joined, domidumont left
jnthn One more op to re-work, and then it'll be on to debugging 13:15
13:15 domidumont joined
jnthn ah, ran out of time before Czech class. Will finish it up after :) 13:30
timotimo have fun 13:32
good learn
lizmat sanity check: supporting static libraries is not possible, nor likely in the near future, right? 13:40
(as in NativeCall)
timotimo statically linked, you mean? 13:41
i think you just have to do it the same way you get libc functions, i.e. "is native(Str)"
lizmat so you're saying it *is* supported already
timotimo you'll "just" have to link them statically into your moar executable
lizmat ah, ok
timotimo: actually, that's even "is native" without anything nowadays 13:42
timotimo oh, cool
that's theoretically also how you get moarvm functions, but since you don't have access to the underlying objects for everything, all you can do is embed a moarvm inside a perl6 program
which is, of course, also a fun thing to do
haven't tried it yet
lizmat well, eh, I guess you have more important things to attend to :-) 13:43
timotimo i should be doing more stuff :|
well, grant's progressing, that's nice
i'll soon have to select a graphing library to make some pretties 13:44
lizmat looks forward to a progress report :-)
dogbert2 timotimo: btw, didn't you find some strange nullbytes in the telemetry log the other day? 13:51
brrt lizmat: we still all want a statically compiled 'perl6' binary, and preferably a perl6 --compile option as well 13:53
lizmat yeah, but that is pretty far down the list, no ?
timotimo i think it was actually the speshlog
why?
ooooh, fontawesome has a set of icons called "stroopwafel" 13:55
brrt that depends on somebody making it a priority and implementing it
dogbert2 timotimo: isn't it something which we should fix?
timotimo it was a user error i'd claim 13:56
dogbert2 :)
brrt I'm going to be a grouch about this for a second, but imho making a p6ake make-alike was (and is) not a good way to push perl6 forward
dogbert2 can't fix those
brrt I mean, if you like to write another make, be my guest for sure. Should that be in the official perl6 repo (as an 'endorsed' project)? Maybe not 13:57
timotimo oh i didn't realize it lived under the perl6 org
i think it's harmless, though
brrt it's harmless until people start believing that it is *the* perl6 build tool 13:58
and yeah, it's senseless to tell people 'don't do this, do that instead' if they are volunteers 13:59
(
the other aspect is that a perl6-specific build tool makes users about as happy as any other language-specific build tool 14:00
which is to say, not very happy at all 14:01
14:03 zakharyas left 14:04 zakharyas joined 14:27 geekosaur left 14:28 brrt left 15:03 zakharyas left 15:04 zakharyas joined
jnthn back 15:17
Urgh, I thought the inscrutable/undecidable/impossible style words were going to be nice and regular in Czech, unlike all the different prefixes in English. Turns out that there is a consistent prefix and ending. But the verbs get mutated in the middle in various not really regular ways. :) 15:19
dogbert2 Jnthn je zpátky
jnthn :)
Waterbed principle strikes again... :P
15:20 robertle left
dogbert2 isn't that a timtoady classig 15:21
*classic
jnthn I heard it from TimToady, yes :) 15:26
Geth MoarVM/ctx-lazy: 4b369ea7f2 | (Jonathan Worthington)++ | 7 files
Make getlexrel traverse the MVMContext
 15:35
MoarVM/ctx-lazy: 379bbaa096 | (Jonathan Worthington)++ | src/core/interp.c
Switch to using MVMContext traversals

With this, we just store up outer/caller displacements against the base context, and then interpret them each time using the frame walker. This allows us to do the right thing, even in the face of inlining and/or later deoptimization.		 15:39
jnthn One NQP test failing. Currently seeing if it'll build Rakudo. 15:40
Geth MoarVM/ctx-lazy: 51a45614ac | (Jonathan Worthington)++ | src/spesh/frame_walker.c
Fix thinko in caller skip thunks traversal
 15:47
jnthn That fixes the NQP test :)
Rakudo built but failed lots almost certainly due to that very same issue :)
15:50 brrt joined 15:57 robertle joined
Geth MoarVM/ctx-lazy: 7906b5d6f6 | (Jonathan Worthington)++ | src/6model/reprs/MVMContext.c
Fix an off-by-one
 15:58
15:59 zakharyas left
jnthn Eww...loads of really weird crashes 16:00
16:00 domidumont left
brrt :-( 16:15
jnthn GC debug mode points to an illegal gen2 -> nursery in the inter-gen set 16:21
oops
*not* in
In the call chain, after a heap promotion
Hmm, all the things that SEGV complain about that 16:25
Hm, I wonder if it's some kind of mis-compile 16:27
16:32 geekosaur joined, geekosaur left
jnthn Hmm, even compiling NQP fails (in an odd way) with GC debug mode turned on 16:35
16:36 geekosaur joined
dogbert11 could there be a bug in the code which is run when MVM_GC_DEBUG is turned on? 16:38
i.e. in the GC error detection code
16:38 brrt left
jnthn Looks like 7314e231055d37f is guilty of the NQP build regression under GC stressing 16:53
17:05 lizmat left
jnthn But I've no idea why 17:12
But it looks memory corruption-y 17:13
Hm, or maybe just wrong result-y
17:22 domidumont joined
jnthn By now discovered it only goes wrong if inlining is enabled 17:25
ooh, it's also JIT sensitive 17:29
And indeed, with the JIT on it determines a different set of inlines as applicable 17:36
And that's 'cus a bunch of the time it returns -1 (didn't find a deopt index) instead of the appropriate deopt index 17:38
jnthn wonders why that'd fail 17:42
It's for a frame that's not tc->cur_frame, so should always use jit_reentry_label
Hm, and it does sometimes work, just not always 17:43
ah, and this particular issue happens without a small GC nursery, though with spesh blocking on 17:46
Also the times when it works are also JITted too 17:49
dogbert11 sounds a bit like the stray 'break' statement you found the other day 18:02
jnthn That's a different isuse, alas. 18:04
OK, I understand half of it
github.com/MoarVM/MoarVM/blob/mast...aph.c#L732 18:06
Under nested inlining the DEOPT_ALLs turn into DEOPT_INLINE
But even disabling nested inlines doesn't fix my issue
And most of the deopt_idx from the JIT seem off by one relative to the interpreter ones 18:07
And there's still some that fail to be found 18:08
dogbert11 are all lazy_ctx changes in Moarvm? 18:10
jnthn Only in the brach
Though this particular issue is introduced quite a few commits back from that branch
dogbert11 aha
jnthn uh, from the tip of the branch
dogbert11 checked out the branch and built Moarvm, but building nqp failed 18:11
Local 'pkg_lookup_tmp_33' already declared
at gen/moar/stage1/QAST.nqp:4855 (gen/moar/stage1/QAST.moarvm:register_local)
jnthn Yup
That's exactly what I'm hunting 18:12
18:13 lizmat joined
dogbert11 ok 18:13
I can add that asan is silent 18:14
jnthn Anyway, the original code I replaced did the JIT lookup using a different way of determining what inlines we are inside of 18:17
I was hoping to unify the code paths since I expected the deopt index to be reliable, but it ain't
One reason I can see, but the rest I can't 18:18
Maybe I'll just re-work it to use the other approach to finding what inlines we're in to solove that for now 18:20
But for now, time for dinner 18:21
18:47 brrt joined 18:51 Ven`` joined 19:08 domidumont left 20:27 Ven`` left 20:30 brrt left, brrt joined
jnthn brrt: Dunno if you saw my JIT deopt point ponderings above and have any ideas. :) 20:34
brrt i did not see them yet
jnthn np 20:35
brrt oh, hmm
that's odd
i'll take a look at what you're doing now and see if i can spot something off
jnthn Yeah 20:36
I'll be back in 30 minutes or so :)
brrt ok
:-)
we have, like, a lot of open pull requests 20:53
20:53 Ven`` joined 21:08 Ven`` left
jnthn back 21:09
brrt \o 21:10
i have no idea tbh jnthn
jnthn OK, then I guess it'll be an extended bug hunt, and my most expedient bet is to use the jit applicable inline functions 21:11
brrt :-(
jnthn Which isn't bad per se
brrt does the jit bisecter help?
jnthn In that I'll be adding this in the frame walker, which is now used from everywhere
Didn't give that a try yet 21:12
brrt hmm, I hope it works 21:15
jnthn tbh, I'm already so deep in solving genuine problems that need to be solved but aren't actually the thing I want to deliver that I'm OK with not trying to unify something that doesn't urgently need it :) 21:16
21:24 robertle left
Geth MoarVM/ctx-lazy: 1518d0ac80 | (Jonathan Worthington)++ | src/spesh/frame_walker.c
Use JIT inline mechanism over deopt indexes

The deopt indexes are often either missing or seem to be off-by-one when we ask the JIT for them. That's probably ungood for deopt itself, but for our purposes here we can use the JIT inline position interface.		 21:33
jnthn brrt: The deopt inline mechanism worked fine for this
brrt: Except now that the frame walker knows it, it'll be used in lots more places we need to be aware of inlines :)
So now NQP is clean under MVM_SPESH_BLOCKING=1 with the branch 21:34
Next to see what happens with Rakudo
brrt i'm definitely missing the necessary brainpoints for useful comment 21:37
but it looks sane :-)
jnthn Yeah... My problem now is that whlie Rakudo built earlier, it also had utterly weird segfaults 21:38
And other bizzare failures 21:39
Darn, looks like "still there" 21:40
So now to bissect my branch to find out where the breaking point is
ooh, not a SEGV, just a wrong result now 21:41
brrt less bad
jnthn Yeah, and less fails in make test 21:42
And one of the failures is...exactly the error I was trying to fix in the ANTRLR module
That almost certainly means some bug in the context traversal 21:43
Yeah, just aborted spectest at S05 but a lot less problems than earlier 21:49
Realized that immediately when we manually traverse, we should not when starting to iterate move forward a frame from there; we were already at the right place to start 21:52
And fixing that fixes Rakudo make test. Hurrah. 21:53
brrt \o/
jnthn Into the S03 tests; t/spec/S02-types/range.t is the only failure so far, but I didn't pull Rakudo HEAD and I'm sure a fix went in for some range thing recently recently. 21:57
Geth MoarVM/ctx-lazy: d31cd0f10f | (Jonathan Worthington)++ | 2 files
Don't miss frames that we traversed too

We iterate by calling the frame walker's "move next" operation in a loop. However, immediately after a manual traversal, we've already
  moved ourselves to the first frame we want to consider. Thus, never
skip over it.		 21:59
jnthn Into S06 (slow because my home machine + VM, and spesh blocking) 22:02
And looking alright
brrt hmm, my expression tree refactor keeps on screwing up the tree structure :-(
confusing nodes and operators
operator nodes and links, i mean
jnthn :-( 22:05
brrt off by one somehow
the objective of it all is that we can modify the tree and keep everything contained in the single array, which I'd think make optimization easier 22:06
proving to be harder than I thought
timotimo jnthn: how would you expect the "beginnings of gc runs" times to be graphed?
brrt anyway, afk tonight 22:07
jnthn 'night, brrt
timotimo gnite brrt
jnthn timotimo: Hmm...what are we wanting to understand, how often GCs happen? 22:09
Could do it as a timeline with a point for each GC
timotimo i'd expect "are there irregularities in distances" to perhaps be interesting
like, can you see the program go through different phases, or are there hot spost 22:10
hot spots*
that was my first idea
i could imagine putting lines instead of points, to show the duration, but the duration is usually extremely short compared to the distance
22:11 brrt left
jnthn Maybe use horizontal space for interval between them, vertical for duration, and color for nursery/full? 22:13
timotimo mhh, interesting, let me have a look what the library i'm using gives me 22:14
jnthn The good news is that ctx-lazy does appear to have fixed the issue afflicting ANTLR
For some reason unknown to me so far, it doesn't fix P5-quotemeta 22:15
About to get ctx-lazy spectest results too 22:18
Yay, looks decent. 22:20
timotimo jeez, i blew my 16 gigs of ram again 22:24
22:41 MasterDuke joined 22:44 lizmat left
jnthn Argh, I just figured out what's wrong :( 22:57
timotimo uh oh, frownie face after that sentence isn't good 22:58
jnthn It's too lazy
22:59 hoelzro left, leedo left, huggable left 23:00 buggable left
timotimo so something gets thrown away because it doesn't keep enough stuff alive? or something? 23:00
jnthn When we have no inlining, the caller chain is effectively immutable 23:02
So just keeping the tip is enough, and we could lazily traverse no problem
timotimo ok, and inlining deopt makes that no longer valid? 23:03
jnthn But when there's inlining, what we actually have is a frame that is moving between different inlines over its lifetime
timotimo ah, so if we go from one inline to another, or deeper into one, we don't yet have a way to figure that out? 23:04
jnthn Hm, actually it still doesn't really make sense. 23:05
Argh, right. The return address lives in the caller 23:06
And isn't snapshotted in the callee
Anyway, when we have inlines we no longer have the immutable linked list 23:10
So if there's some time between us doing nqp::ctx and nqp::ctxcaller and/or an actual lookup, stuff can mutate under us 23:12
So what we'd need to have is a snapshot of how we were reached 23:14
I *think* that each frame that has a callee that does inlining could just store the deopt_idx (interp) or current position (JIT) 23:15
And then we use those in the traversal
And we only have to look down the stack to the point that we see a frame where we already computed this information and can go no further 23:16
So it doesn't have to be terribly expensive for deep stacks
Plus we can only compute this at the point we make a context
Can stick it in the frame extras so it won't make frames bigger if the feature ain't used 23:17
timotimo mhm
23:33 lizmat joined 23:59 buggable joined, ZofBot joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!