01:48 ilbot3 joined 01:57 zakharyas joined 05:04 lizmat_ joined 05:40 domidumont joined 05:44 domidumont joined 06:03 domidumont joined 06:16 lizmat joined 07:13 tokuhirom joined
dalek arVM: 62ef65c | jnthn++ | src/gc/worklist.h:
Fix typo in debug output.

  "Null STable in time" would make a good song title, though...		 07:59
arVM: a183e38 | jnthn++ | src/profiler/heapsnapshot.c:
Teach heap snapshotter about frame collectables.
 08:02
arVM: 85375aa | jnthn++ | src/profiler/heapsnapshot.c:
Don't put callstack frames on snapshot worklist.

We'll visit them specially, as the GC itself does.
arVM: 15b6a0f | jnthn++ | src/profiler/heapsnapshot.c:
Ensure we always produce a heap snapshot.

Force a GC run at the end of heap profiling, so we get an "exit snapshot". This also means that we'll never spit out empty snapshot sets if we snapshot a program that runs for a very short time (as we might when examining the base memory use of NQP/Rakudo).		 08:04
timotimo aha! i implemented a183, but i didn't think of the changes in 85375 08:06
dalek arVM: 088dc71 | jnthn++ | src/profiler/heapsnapshot.c:
Walk the correct thread's inter-gen root set.
arVM: 6805576 | jnthn++ | src/profiler/heapsnapshot. (2 files):
Include thread local callstacks in snapshots.
 08:22
jnthn timotimo: I pulled various of your heap analyzer improvements into mine also :) 08:43
timotimo ah, neat 08:45
the implementation of "more" is less than awesome, but it surprisingly works! :P 08:46
jnthn That seems to get it working nicely again and lets you see what was on the thread local call stack at each snapshot, anyways
08:46 zakharyas joined
dalek arVM: 494661e | jnthn++ | src/profiler/heapsnapshot.c:
Remove "we're snapshotting now" debug output.

Don't really want to dump it all over the user's program output.		 08:47
jnthn Yes, I didn't take `more` for now 'cus of the code dupe and also it's got at least one bug (keeps returning the last page of results when you hit the end!) :)
timotimo ah, hehe 08:49
yeah, i'd rather re-implement it in a saner fashion
maybe gather/take would actually make sense there
08:50 ilmari joined 08:58 MadcapJake joined
jnthn timotimo: Do you plan to look further at the utf8-c8 thing at some point? 08:58
timotimo i could try 08:59
the way it works messes with my brain :P
jnthn :) 09:07
timotimo yeah, forgot to "make install" 09:08
that's not going to help
jnthn OK, will leave it with you for now and work on other bits 09:10
Though going to do my other MoarVM perf bits today in branches, because The Release Is Soon
timotimo twiddles lots of < vs <=, + 1, - 1, ... :P 09:14
09:40 brrt joined
timotimo i seem to have stumbled upon something that works. now to stress-test it. 10:10
jnthn seems to have gotten a couple of seconds off Rakudo's build time 10:16
timotimo cool. it's not even parallelized! :)
i've not fixed the thing, but it at least doesn't invalid-read-of-size-1 any more 10:18
dalek arVM/jnthn/opts: ff710c7 | jnthn++ | src/ (6 files):
Avoid VMNull setup memcpy/loop in spesh'd frames.

Just shove in `null` instructions for all object registers when we compute the CFG, but before forming SSA. Then, let dead instruction detection rip out all that we don't need, so only read-before-use registers get the VMNull put in them, and we do nothing for the others.
timotimo nice
arnsholt Yeah, definitely! 10:39
brrt \o/ 11:03
very clever... jnthn++
12:12 zakharyas joined
dalek arVM/jnthn/opts: 6ada706 | jnthn++ | src/core/exceptions.c:
Use existing function, to ease upcoming refactor.
 12:13
jnthn is really quite confused about how his Rakudo build is busted 12:40
Apparently, by above patch
timotimo the in_caller_chain uses the wrong tc perhaps? 12:41
timotimo didn't look closely enough at all
nwc10 will test once valgrind has finished what it's currently doing
jnthn It doesn't look at tc at all
timotimo ah, hm
jnthn It's literally this:
static MVMuint8 in_caller_chain(MVMThreadContext *tc, MVMFrame *f_maybe) { return f_maybe->tc ? 1 : 0;
}
timotimo shrugs 12:43
jnthn Going back one before that commit fixes things
timotimo must be some nondeterministic problem that just gets shuffled around?
jnthn Well, could be I guess but...wtf 12:44
timotimo yes wtf
nine jnthn: rakudo builds just fine here with that patch 12:47
jnthn um, and it did here too after I went back to it o.O 12:48
timotimo o_O
jnthn ...wonder if I had a really weirdly busted build
timotimo time to buy a new machine. oh and tear down your apartment building and build it anew
moritz yes, give the Computrons some time to recover 12:49
jnthn I think some repair guy did that to our boiler today and I'm not sure it's better off now :P
timotimo the magic smoke may have been shook too hard somehow?
jnthn It's really, really odd. A seemingly innocuous change causes a bizzare CORE.setting compilation failure 13:24
(Default invocation handler is not invokable) 13:25
nwc10 use more valgrind?
or GC bug?
jnthn Disabling JIT/spesh doesn't hide it
Not sure.
It's hard to tell
dalek arVM/jnthn/opts: 8ed7ffc | jnthn++ | src/ (4 files):
Stop caching MVMContext on a frame.

We rarely request them anyway, and it should be fairly uncommon to ask for it more than once. And even then, it's cheap to create, and while it costs an extra allocation or so we already are using less space in the nursery for all the other frames (due to losing 8 bytes thanks to not needing the cache slot), so will very likely come out ahead even in those cases.		 13:28
jnthn That's the commit that triggers it
Building in my Linux VM too now, to see if it happens there
Yup, I get it there too 13:32
valgrind time 13:33
valgrind: the 'impossible' happened: Killed by fatal signal 13:34
o.O
masak jnthn -- makes the 'impossible' happen in code, sometimes in a good way! :P 13:36
jnthn It manages to spew this out: 13:37
Thread 1: status = VgTs_Runnable
==915== at 0xD2E5A05: ???
==915== by 0x502B668: MVM_jit_enter_code (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
And the usual stuff below
Trying again with MVM_JIT_DISABLE=1... 13:38
But that didn't help locally
uh, off the VM I mean 13:39
Curious if anyone else building the branch gets a similar failure. 13:41
With JIT off, valgrind is happy...and it explodes 13:42
moritz building now 13:48
nwc10 Stage parse : Default invocation handler is not invokable
ASAN says nothing
jnthn Yup, that's the one
I struggle to see how the patch in question could lead to that...
moritz I get the same error as nwc10 13:50
jnthn Running both a valgrind with GC debug turned on and FSA debug turned on, and also a build on Windows with a small GC threshold, to see if either of them give a clue. 13:52
Really weird.
The first of those two didn't show anything different. 13:55
Nor did the second 13:59
JimmyZ How about disabled spesh too ? 14:00
jnthn Already tried that. No difference. 14:01
JimmyZ tries build rakudo with moarvm master and only with that patch 14:03
jnthn That combo I didn't try
Without that patch things seem fine
JimmyZ ok build a core setting 14:04
but ./perl6-m tools/build/install-core-dist.pl /home/cloud/OpenSource/rakudo/../MoarVM/install/share/perl6
getlex: outer index out of range
jnthn hmm 14:05
JimmyZ hmm, looks like this error is not about that patch 14:06
without patch I also get this error
but I build a core setting with that patch 14:07
jnthn That's even more odd...
The only thing that branch has over Moar master is a spesh patch (but disabling spesh makes no difference whatsoever) and re-using an existing function that's really simple 14:08
Stranger yet, if I log when we re-use contexts while building CORE.setting, to see where we're allocating another one...that code doesn't seem to fire 14:13
Not until stage mast, anyway 14:14
ohhhh 14:15
Darn it.
Serialization relies on them being interned.
And that's why it seemed so action-at-a-distance 14:17
moritz serialization-at-a-distance
JimmyZ oh , that's why I buld a core setting, beacase i use a old *.moarvm code to build the CORE setting. 14:18
I only rebuild the CORE setting. 14:19
and make realclean I got the same error
jnthn aha :)
yeah, guess that's why I had funny stuff before too
Interesting. I didn't realize serialization did that :) 14:20
Or forgot at least
JimmyZ and the other error I see because I use a new CORE setting :P 14:21
jnthn Now that frames themselves are MVMCollectable, they have an SC slot...so we could probably eliminate the use of the context wrapper altogether during serialization 14:26
We also do some linear scans of context lists 14:29
Seeing how long... 14:30
nine I'm all for everything that makes serialization simpler ;) 14:33
JimmyZ jnthn: the pacded SC + object_data_offset is about 1MB in the CORE setting size , FYI :)
*packed 14:34
which is use 8 byte per object 14:35
and it is big int and can't use varint 14:36
jnthn Yeah, 'cus we need to be able to index into that table :) 14:37
JimmyZ yeah, and another part about SC don't have a index 14:38
which uses varint now
jnthn iirc, we also only look at that table rather than copying it into memory 14:39
In which case it's mmap'd and shared over all processes 14:40
JimmyZ aye
jnthn So it's not quite so bad
JimmyZ yes, just FYI ;)
jnthn Yeah, it's interesting to know :) 14:41
In CORE.setting compilation, that thing I mentioned us linear scanning can get up to around 1600 things long
So not ideal but not a huge deal
But still, I'd not mind getting rid of using context wrappers there 14:42
Hm, seems like an overall simplification. :) 14:58
Now to see if it'll work
JimmyZ Segmentation fault 14:59
jnthn :P
It passed the serialization tests in the nqp repo at least :) 15:00
The NQP build is good, Rakudo one is working on CORE.setting 15:02
it made it :) 15:04
So question is...what if I try it on top of my context caching removal patch :)
JimmyZ still a good patch to push to master ;) 15:07
jnthn This lot will all make it into master shortly :) 15:08
Just not before the monthly release :)
Since they're all optimizatins, and commiting those on the eve of a release is kinda needless risk :)
*optimizations
JimmyZ yes
jnthn I think the Rakudo release is scheduled for tomorrow, so I'll probably do the Moar release this evening, 'cus I may end up a little busy tomorrow. 15:09
hoelzro jnthn: I thought the rakudo release was on saturday? 15:11
I hope so, since I'm the release manager this time =) 15:12
I wouldn't mind a MoarVM release as a head start, though!
jnthn hoelzro: oh... :)
Right, I still had Thursday in my head
If you're release manager then it's whenever you want it ;-)
I'll probably hold off the Moar release until Friday in that case, though today's changes can still wait until after it. 15:13
But gives chance to fix any urgent things noticed in the next day or so
Hopefully nothing :)
yay, everything built :) 15:14
hoelzro Friday works for me! 15:27
I'll probably do most of the work for it on Friday night my time
so very early Saturday for $everyone-else (-) $other-americans
15:33 tokuhirom joined
dalek arVM/jnthn/opts: 6329eee | jnthn++ | src/6model/s (3 files):
Use frames directly when serializing closures.

Before, we obtained a context ref, just so we could mark the SC on it. However, now that frames are collectables we can mark an SC directly on them. This avoids a bunch of allocation and indirection. It also eliminates the need to have context objects be interned, which was removed in the previous patch.		 15:48
jnthn m: say 9152 / 9176 15:50
camelia rakudo-moar e39ce3: OUTPUT«0.997384␤»
jnthn A little bit off NQP base memory
m: say 46808 - 46736
camelia rakudo-moar e39ce3: OUTPUT«72␤»
jnthn 72KB off Rakudo's too
But since it makes frames a little smaller it also means a tad less GC pressure also :) 15:51
s/also$//
Enough for now :) 15:58
16:34 tokuhirom joined 17:03 domidumont joined
nwc10 jnthn: I assume by its aloof silence that ASAN considers your fix to be tolerable. 17:59
jnthn: I assume by its aloof silence that ASAN considers your fix to be tolerable. 18:02
er, oops, that "up" was supposed to be for the other window (with the build)
18:49 vendethiel joined 19:44 Ven joined 19:58 camelia joined 20:01 MadcapJake joined 20:36 tokuhirom joined 21:00 tokuhirom joined 22:01 vendethiel joined 22:37 vendethiel joined 23:01 tokuhirom joined 23:32 vendethiel joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!