github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
07:04 patrickb joined, domidumont joined 07:09 robertle joined 08:26 harrow left 08:36 harrow joined 10:04 zakharyas joined 10:05 domidumont left 10:47 sivoais joined 11:16 zakharyas left 11:49 brrt joined 12:00 lucasb joined 12:11 domidumont joined 12:32 brrt left 12:35 zakharyas joined, sena_kun left, sena_kun joined 12:43 brrt joined 12:51 brrt left
Geth MoarVM: cfb5b3ae0f | (Jonathan Worthington)++ | 2 files
Put limits on local and inline counts for inlining

We have some non-linear algorithms as part of inlining - of note, in the deopt tracking. If the frame gets too big in various ways, the cost of that can become very large. Put some limits in place to catch such cases, so we won't end up spending a vast amount of time trying to win performance, but end up blocking the application for multiple seconds due to the spesh thread getting stuck in a highly costly loop.		 13:08
nine jnthn: that must have been very interesting frames 13:12
jnthn The routing regex generated by Cro::HTTP::Router for an application with quite a lot of routes :) 13:14
"regex" - it contains a ton of code blocks too
And they're mostly all small assertion checks, so it decided to try and inline them all, which would be fine, except for the non-linearities involved. 13:15
The innermost part of the loop was ending up doing half a million executions per inline 13:16
I suspect there's a better way to deal with this, but for now this avoids doing such costly work 13:17
lizmat fwiw, I've seen it take 20+ seconds :-) 13:18
jnthn I guess what makes the deopt so active is that a regex has some very long-lived registers too (like the current position)
So they end up living over a load of deopt points, also making the analysis very costly
lizmat: I didn't time it, but I think it was heading that way for me too 13:19
lizmat it took so long, most people assumed it was hanging permanently 13:20
nine Wow....so this is the first time spesh's own performance became a real issue?
jnthn Well, in such a dramatic sense, yes 13:21
We did also have a case where we got a significant increase in spesh cost, also related to deopt tracking
Though that wasn't causing anything like this, just a measurable slow-down in spectest time
(And the algorithm in question got optimized quite a bit later.) 13:22
timotimo huh, but spesh normally won't block the program? 13:31
jnthn It needs to join GC 13:33
timotimo ah, right 13:36
it occurs to me that a system that sends notifications of things like "gc is happening" to an observing process wouldn't be able to send a "gc started" signal until the gc has finished 13:41
and a situation where gc is taking very long wouldn't be identifiable as such 13:42
jnthn Well it would be when we finally get around to it...it's not that the thing hangs forever 13:45
So it'd belatedly show what happened
timotimo if we ever have a bug that gets the spesh thread busy for like two to five minutes, an impatient dev may have already killed the process and be none the wiser 13:46
lizmat impatient devs have done so after 5 secs already 13:47
14:18 lizmat left
Geth MoarVM: 2d54fa36bc | (Jonathan Worthington)++ | src/profiler/log.c
Fix dealloc profile logging

The type object itself may also have been moved, and thus we should make sure to follow any forwarder. Presumably, we'll have seen its allocation during profiling, so the type object will not be going away.		 15:01
timotimo ooh, good catch 15:08
that's probably the cause of these crashes i wasn't able to figure out a couple weeks ago
jnthn Not the only profiler problem, alas... 15:10
Running into various "profiler lost sequence"
timotimo yes, those puzzle me rather a bit; maybe i can come up with a semi-graphical tool that lets me more easily explore what's going on there 15:11
today is just a tiny bit warmer than it used to be and i'm already falling apart figuratively
15:20 sena_kun left, sena_kun joined 15:21 sena_kun left 15:26 pamplemousse joined 15:41 robertle left 15:52 patrickb left 16:00 domidumont left
jnthn Think I've figured out various of the crashes that show up when we do multi-threaded profile / debug 16:22
Still didn't figure out the lost sequence though :( 16:23
16:23 Kaiepi joined, lizmat joined
Geth MoarVM: ab7bac5ad1 | (Jonathan Worthington)++ | src/core/frame.c
Avoid various races in profile/coverage//debug

We could attempt to duplicately instrument code when multiple threads started to execute it around the same time, which could lead to crashes, since it was assumed that would only happen once. Do it under lock, so as to prevent such issues.		 16:39
timotimo ooh, ouch
i was somehow under the impression that racing to install an instrumentation was fine, i guess?!
jnthn We still have issues with the sequencing, alas...and also debug, which seems to SEGV or die with random errors 16:42
(On the large app I'm testing it with)
16:44 zakharyas left 16:49 lizmat left
timotimo the debug instrumentation, right? 16:50
using them at the same time is unfortunately not possible, though i've wondered if maybe there should be a way to get the debugserver without the breakpoint instrumentation
jnthn Yes, the debug one 16:51
timotimo i haven't had that one crash at all :o
jnthn Also odd: I've currently got a deadlock under it 16:52
Wherein we have a lot of spesh plugin call_resovler instances trying to uv_mutex_lock(&tc->instance->mutex_hllconfigs); 16:53
And they're deadlocked on that
And if I look in the lock structure to see who owns it...it's one of the threads waiting for it
16:57 patrickb joined 16:58 Kaiepi left 16:59 robertle joined
jnthn OK, this is very odd 17:27
(Separate from the lock issue above) 17:28
We have a case where the instrumented frame tries to execute an op that does not exist in the instrumented bytecode
timotimo some reentrancy issue or are other locks involved somehow?
jnthn the op is atpos2d_n, fwiw
Which is op number 665
Which appears *nowhere* in the disassembly
But... 17:29
00058 breakpoint 368, 665
timotimo could very well be the cur_op went sideways half an instruction
jnthn Yes, seems to...but...how?
timotimo that happened back when we still uninstrumented code after turning off the profiler
jnthn This is debug, not profile
Also, I didn't actually connect a debug client
timotimo the instrumentalization is now properly locked, yeah? 17:30
any deopt happening that could be screwing up?
there's no possibility to inline a frame that hasn't been properly instrumented for example when a frame calls that frame only very rarely, and gets speshed before it does that one for the first time? 17:31
jnthn I think that locking is done properly, yes
There's no spesh cand on the frame
Further, the static frame has no spesh candidates yet 17:32
So I think deopt of this frame is impossible
The frame itself *is* being invoked in parallel 17:33
Oh...uh...
Dammit, we were updating the instrumentation level *before* performing it
So yeah, we could end up doing nasty things I guess
timotimo ouch 17:34
wew that's dangerous
17:35 rfold joined
jnthn Isn't it jsut 17:35
*just
jnthn crosses fingers, runs again 17:36
17:37 lizmat joined
jnthn Yeah, I've never managed to get this app this far under the debugger 17:38
Nice
Yeah, it's holding up pretty good so far
timotimo amazing that it'd take us this long to find it, though instrumentation isn't that common for regular users
jnthn Even better news: it fixes the broken profiling I had earlier too :D 17:45
timotimo brilliant!
jnthn I wonder if it fixes the fact that the debug server immediately falls in a heap on Windows even on simple programs...I fear not 17:46
timotimo o_O
jnthn OK, will test/spectest
17:46 lizmat left
jnthn If you have a moment to look into why it explodes on windows, that'd be nice :) 17:46
timotimo i wonder if one of those 1h windows VMs from appveyor would be enough for that :) 17:47
jnthn The profiler issue was a "lost sequence" one, by the way 17:52
I think probably the race meant that we got code installed that did an exit, but missed the enter
timotimo quite possible, though the breakpoint ops would also mean some ops get executed twice maybe. that's not quite as dangerous as skipping ops, though 17:53
jnthn Well, op boundaries moving was very dangerous :)
timotimo aye 17:54
jnthn But anyway, this makes debug/profile a lot more useful for multi-threaded apps :)
As in, "not liable to fall in a heap" :)
timotimo <3
Geth MoarVM: c3c9d7dfac | (Jonathan Worthington)++ | src/core/frame.c
Bump level until after instrumenting

Rather than before, which can lead to various races. This fixes a whole range of different problems in both debugging and profiling of apps that are multi-threaded.		 18:00
jnthn dinner & 18:15
18:54 patrickz joined 18:55 Kaiepi joined 18:57 patrickb left 19:02 MasterDuke joined 19:14 lizmat joined
MasterDuke in case anyone is curious, a perf record of install-core-dist.p6 now shows merge_graph at the top 19:17
yoleaux 17 Jul 2019 22:48Z <jnthn> MasterDuke: It looks like a handle isn't being closed somewhere at process termination; I don't know where without spending some time debugging it, but there's a decent chance it should be happening in MoarVM somewhere
19:25 sena_kun joined
timotimo merge_graph from spesh? 19:26
if so, a spesh log could be interesting; find which has the longest time taken
19:33 MasterDuke left 19:38 zakharyas joined
jnthn MasterDuke: Does it still show that after today's patch? 19:40
dogbert17 MasterDuke: I updated R#3057 with some info about what kind of fd is lost 19:44
synopsebot R#3057 [open]: github.com/rakudo/rakudo/issues/3057 "too many open files" when repeatedly using Proc::Async
19:45 MasterDuke joined
MasterDuke timotimo: yes. i tried to get a spesh log, but there are multiple processes, so i get some `free(): corrupted unsorted chunks` and then an incomplete/corrupted spesh log 19:46
jnthn: yeah, at HEAD as of right before i posted 19:47
timotimo: i think i've patched moarvm before to add the pid to the filename, i can do that again 19:48
19:48 lizmat left
jnthn MasterDuke: I think if you do something like spesh_log.%s or some such it will subsititute the process iD there 19:49
MasterDuke jnthn++ i'd forgotten about that 19:51
timotimo aye, it wants a %d 19:54
MasterDuke ok, now have 14 log files. but what am i looking for the longest of? the string 'merge_graph' doesn't show up in any
timotimo right, you'll have to find the one with the longest total time
i'd grep out all "time taken" first, and sort them
then grep for the top results to find the files they come from
and from there, find the last "After" before that, or maybe look below for the "latest guard tree" or whatever 19:55
MasterDuke longest specialization was 168474us, there were 6 > 100000us 19:58
timotimo i'm not so good with units, how many seconds is that? 19:59
1000 us == 1ms?
1000000 is one second?
so the longest specialization time there was 0.17s? 20:00
MasterDuke `Spesh of '' (cuid: 294, file: gen/moar/stage2/QAST.nqp:5682)` 20:01
400 BBs
timotimo that's a big one. i wonder how many of the inner blocks from the ifs and fors have been inlined at nqp's optimizer stage already 20:02
elsif $ins_result_kind == $MVM_reg_int32 || $ins_result_kind == $MVM_reg_int16 || 20:03
$ins_result_kind == $MVM_reg_int8 || $ins_result_kind == $MVM_reg_uint64 ||
$ins_result_kind == $MVM_reg_uint32 || $ins_result_kind == $MVM_reg_uint16 ||
$ins_result_kind == $MVM_reg_uint8 {
MasterDuke believe it's github.com/perl6/nqp/blob/master/s...T.nqp#L988
timotimo you think this gives very big bytecode? big enough that we want an nqp::list_i instead? or something?
yes, that's the line for me locally, too 20:04
MasterDuke i have no idea. sounds like a jnthn question
timotimo every || will turn into a conditional
i.e. increase the number of BBs 20:05
MasterDuke ah
would it help to pull something like github.com/perl6/nqp/blob/master/s...1165-L1198 out into it's own method/sub? 20:06
timotimo good question; i'd say the vast majority of registers have obj, int64, num64, str, or perhaps even uint64 20:08
so perhaps put only all the elsifs out there
though i must repeat, i don't know how much bytecode this part translates into
though i guess looking at the line numbers in the BBs could be interesting 20:09
jnthn Could just make an array and index it with the register kind, I guess 21:06
Or a bitmap if they're powers of 2
21:06 brrt joined
brrt \o 21:06
pamplemousse o/ 21:07
brrt pamplemousse: how are things going?
anything I can do to make the process smoother? :-) 21:08
pamplemousse Things are going well! I've been working on adding the --compile flag and simplifying things so that to make the exe, you just do perl6 --compile blah.pl6, and to run it, you just type ./blah 21:09
I think I'm probably going to have that done by the end of the day, fingers crossed, which is exciting 21:10
brrt pamplemousse++ that is exciting indeed 21:11
pamplemousse I still have the module packaging to finish, so right now that --compile version would still be calling whatever modules are installed locally on the machine, but I've been doing some reading through things and I'm getting closer to tackling that part as well. I'll probably have a few more questions on the module side of things once I finish up the --compile flag 21:13
21:14 zakharyas left
brrt that's fine, best to do one thing at a time 21:15
how do you like rakudo hacking versus MoarVM hacking? 21:16
pamplemousse Both have been pretty fun. Rakudo hacking is a little interesting since I'll frequently wind up needing to look in the NQP repo or the MoarVM repo for something that's being called to figure out what's going on, whereas MoarVM is a bit more self contained. Working with the Rakudo stuff has done wonders for my tenuous ability to read and write Perl 6, which is nice. I still find myself mostly dealing with calls to MoarVM and 21:22
reading through things here, which I didn’t expect given where most of the actual changes are. I’m enjoying getting to see the bigger picture of how everything works together.
21:22 lizmat joined
brrt :-) it's a big system for sure 21:24
so the executable that you're going to build, it's going to be precompiled, correct? 21:25
(i.e. perl6 source to MoarVM bytecode)
I expect that's going to help a lot already 21:26
pamplemousse Yep! It's MoarVM bytecode. It is being executed via perl6 at the moment, so I'm not avoiding the start up times there yet. But at the very least, it doesn't have to compile anything, which should help at least a bit 21:29
MasterDuke pamplemousse++ i would expect that to help more than a bit 21:33
brrt Cool :-) 21:34
Yeah, the perl6 compiler itself isn't very fast
But like you said, it won't take anything of the cost of 'perl6 -e '''
anything beyond that though, it is likely to help 21:35
21:51 rfold left
timotimo i would expect not having to even look for Grammar.nqp, Actions.nqp, and World.nqp could improve startup a fair bit 21:55
further stuff like the QASTCompilerMAST, QASTOperationsMAST, the optimizers, ...
it really adds up, doesn't it?
jnthn: would you like to link 6guts into the perl6 webring "officially"? it'd not be much more than putting three or four links somewhere and including a javascript if free wordpress allows … probably not 21:56
brrt is there an official perl6 webring? 22:06
are we bringing back webring?
timotimo yes we are!
brrt are we brining back official? :-P
timotimo 6ring.github.io/?
brrt much amaze 22:07
timotimo i'm not sure what official is?
brrt That is very much my point 22:08
timotimo you mean the decentralized nature of the internet makes "official" not be a thing? 22:10
22:10 Kaiepi left
brrt Official is a very vague thing. 22:12
It used to be less vague
and in perl, I guess the only 'official' thing is TPF 22:13
patrickz could someone merge #1131 22:25
brrt what's it about 22:26
timotimo M#1131
synopsebot M#1131 [open]: github.com/MoarVM/MoarVM/pull/1131 Build with spaces in path
patrickz Making it possible to build moar on windows in a path with spaces 22:27
MasterDuke brrt: i haven't gotten around to addressing your comments on github.com/MoarVM/MoarVM/pull/1139, but i think github.com/MoarVM/MoarVM/pull/1142 should be relatively non-controversial
patrickz brrt: It's part of a set of PRs to get rakudo to build and work in a directory with spaces in it. This is the last bit to make the relocatability work actually useful. 22:29
brrt patrickb: I'll have a look, that sounds fairly innocent :-) 22:34
sorry patrickbz
ugh, it's late, I can't type
Geth MoarVM: f0978d6a5f | (Daniel Green)++ (committed by Bart Wiegmans) | src/jit/graph.c
Jit getuniprop_(int|bool|str)
 22:36
patrickz brrt: I'm the same guy.
MasterDuke brrt++ 22:37
Geth MoarVM: 53a6308c46 | (Patrick Böker)++ (committed by Bart Wiegmans) | Configure.pl
Get moar build working with spaces in path

The only bit missing was some quoting to get the build install the include files successfully.		 22:38
brrt patrickz++ MasterDuke++
22:54 MasterDuke left 23:17 brrt left 23:40 patrickb joined 23:44 patrickz left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!