github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:22 MasterDuke joined, p6bannerbot sets mode: +v MasterDuke, MasterDuke left, MasterDuke joined, herbert.freenode.net sets mode: +v MasterDuke, p6bannerbot sets mode: +v MasterDuke
MasterDuke timotimo++ the block was getting re-numbered 00:22
i found out some stuff
the block is entering optimize_plugin, but leaves via github.com/MoarVM/MoarVM/blob/mast...ze.c#L2126 00:24
so the speshresolve (which is there in the before when i look at the right BB) never gets turned into sp_speshresolve, and that then causes the jit BAIL later 00:26
i tried adding some very specific conditionals to get it past that return (e.g., if BB name is 'index' and the opcode is MVM_op_speshresolve), and then index does show up as jitted in a profile 00:28
however, it's significantly slower, because that causes ~800k deopts (instead of the 4 before my changes) 00:29
timotimo oh wow? 00:30
can you see what exactly it deopts on? 00:31
MasterDuke fwiw, when it gets to that return, count is 60, and agg_guard_index_count is 280
timotimo: i haven't checked that yet, how do i tell?
timotimo start with breakpoints in every deopt function, but i don't know how to figure out where exactly in the jitted code that would put you 00:32
and how you'd figure out
does turning jit off make it still do that?
MasterDuke still deopt after my change? 00:33
timotimo i mean can you turn the jit off so you can better see what those 800k deopts are
or does that make the deopt sitaution change? 00:34
MasterDuke i get about 600k deopts with the jit disabled 00:35
now i'll look into the deopt functions
oh, there's a MVM_LOG_DEOPTS, that might prove useful 00:36
hm, not sure how to interpret thes 3.3million line log file 00:39
timotimo isn't it the same output over and over?
MasterDuke yeah
46 Deopt one requested by JIT in frame 'index' (cuid '4053') (302 -> 510)
1127687 Recreated frame 'index' (cuid '4053') 00:40
1127733 Completed deopt_one in 'index' (cuid '4053') with potential uninlining
timotimo the number in the last parenthesis is program counter before and after deopt
should be able to correlate that with the "pc" numbers in the spesh log
and i think in the deopt annotations, too
MasterDuke hm, both pc 302 and pc 510 are in is_narrower, not index 00:45
timotimo that's also why it's uninlining, right? 00:46
MasterDuke i guess, here's a relevant line from the spesh log: [Annotation: INS Deopt Inline (idx 9 -> pc 510; line 272)]
oh, and that is in BB 13 of index 00:47
anything i can do with this info? 00:55
timotimo the instruction after that is probably the one that caused the deopt
MasterDuke prepargs callsite(0x7f83d272c4a0, 1 arg, 1 pos, nonflattening, interned) 00:59
arg_o liti16(0), r9(12)
[Annotation: INS Deopt One (idx 10 -> pc 528; line 272)]
[Annotation: Logged (bytecode offset 520)]
speshresolve r14(10), lits(decontrv)
that's the end of the BB
timotimo i don't know under what circumstance speshresolve would deopt, tbh. though it could be that an op after that does teh actual deopt 01:07
what's the position in the interpreter frame?
MasterDuke uh, how do i find that? 01:09
timotimo "up" until you're in the interpreter 01:11
then "list" for context, or just "frame" for the line number 01:12
01:12 TimToady joined 01:13 p6bannerbot sets mode: +v TimToady 01:17 Ven`` left 01:19 lizmat left
MasterDuke src/core/interp.c:5577, i just put a breakpoint at uninline in src/spesh/deopt.c and this is the first one it it 01:20
timotimo hm, "call MVM_dump_bytecode(tc)" may not help 01:21
but perhaps it does
MasterDuke gist.github.com/MasterDuke17/0c796...185a5d47f5 01:24
timotimo it sometimes doesn't understand where the instruction pointer actually is 01:25
yeah, there is no --> in there at all 01:26
MasterDuke src/core/interp.c:5553 according to the backtrace
which is in OP(sp_guard): { 01:27
timotimo my head aches :\ 01:40
MasterDuke debugging weird deopt stuff is probably not prescribed by 9 out of 10 doctors 01:44
02:24 Kaypie is now known as Kaiepi 04:02 MasterDuke left 04:28 Kaiepi left, Kaiepi joined 04:29 p6bannerbot sets mode: +v Kaiepi 08:18 lizmat joined, p6bannerbot sets mode: +v lizmat 08:35 domidumont joined 08:36 p6bannerbot sets mode: +v domidumont 08:47 robertle left 09:19 robertle joined 09:20 p6bannerbot sets mode: +v robertle
nine Ok, as I understand it, when we deserialize a comp unit we take all repossessions we find there and move those objects to the new owning SC. We later got through this list and add dependencies to all those SCs. This part makes sense to me. 10:14
What doesn't make sense is why apprarently one and only one NQPRoutine from Perl6::Grammar gets repossessed into CompUnit::Repository::Staging in the first place. 10:15
Even weirder, that NQPRoutine is worryobs which is not even used anywhere. When I remove it from the grammar, worry ends up getting repossessed instead.
timotimo nine: something closure related maybe for some reason? 10:38
lizmat feels like an off-by-one error to me 10:49
nine Apparently it's a write barrier hit by the bindkey in this line in quote_lang: my $quote_lang := nqp::existskey(%quote_lang_cache, $key) && $key ne 'NOCACHE' ?? %quote_lang_cache{$key} !! (%quote_lang_cache{$key} := con_lang()); 10:54
Fun fact: directly above this method in the source code there's a: nqp::neverrepossess(%quote_lang_cache); 10:55
So if the stated intent of the code is that this hash is never repossessed and yet we repossess it, I'd say there's a bug somewhere 11:00
Oh and the argument lists in gdb's backtraces shows you the current value of those arguments. Not the values the function was called with. 11:08
timotimo oh, that's good to know
nine Removing the %quote_lang_cache does indeed fix the whole issue! 11:19
timotimo wowza 11:20
nine So I now have 3 workarounds. But at least this last one points at the actual problem 11:24
I think the call to nqp::neverrepossess is just not enough. The cache will be serialized anyway and the flags are probably not part of the serialized data. So we deserialize it during precompilation of a module and happily go on repossess it during the parse. 11:38
timotimo oh, that could totally be the problem here 11:39
nine Maybe this was even the root cause for why I never got in-process-precompilation to work. I gave up when I came across repossession related dependencies that I couldn't explain. 11:41
11:47 kiwi_87 joined, p6bannerbot sets mode: +v kiwi_87, kiwi_87 left
timotimo whoa 11:50
spooky
nine I'd love to just remove %quote_lang_cache or move it to an attribute of role STD, but that'd cost > 10 seconds in settings compilation time :/ 11:51
timotimo then we fix "never repossess" 11:52
or maybe it'd be enough to call that again in the mainline of the containing module
12:52 lucasb joined, p6bannerbot sets mode: +v lucasb 13:31 patrickb joined 13:32 p6bannerbot sets mode: +v patrickb 14:23 MasterDuke joined, p6bannerbot sets mode: +v MasterDuke, MasterDuke left, MasterDuke joined, herbert.freenode.net sets mode: +v MasterDuke, p6bannerbot sets mode: +v MasterDuke
MasterDuke jnthn: did you happen to see any of my conversation with timotimo++ last night? any recommendations for further debugging? 14:24
timotimo: likewise, i'll have a little time today if you have anything else for me to try 14:31
14:33 dogbert17 joined 14:34 p6bannerbot sets mode: +v dogbert17
MasterDuke huh, i just got `free(): corrupted unsorted chunks` when re-running the last part of the install but with a spesh log: `MVM_SPESH_LOG=s.log ./perl6-m tools/build/install-core-dist.p6 /home/dan/Source/perl6/install/share/perl6` 14:38
15:24 bisectable6 left 15:29 shareable6 left, greppable6 left, benchable6 left, undersightable6 left, releasable6 left, notable6 left, squashable6 left, undersightable6 joined 15:30 greppable6 joined, p6bannerbot sets mode: +v undersightable6, squashable6 joined, notable6 joined 15:31 p6bannerbot sets mode: +v greppable6, p6bannerbot sets mode: +v squashable6, releasable6 joined, bisectable6 joined, p6bannerbot sets mode: +v notable6 15:32 benchable6 joined, p6bannerbot sets mode: +v releasable6, p6bannerbot sets mode: +v bisectable6, p6bannerbot sets mode: +v benchable6 15:33 shareable6 joined, p6bannerbot sets mode: +v shareable6 16:00 brrt joined 16:01 p6bannerbot sets mode: +v brrt 16:09 zakharyas joined 16:10 p6bannerbot sets mode: +v zakharyas
brrt \o 16:10
I'm thinking of moving the 3-operand to two-operand form conversion into the register allocator 16:11
probably protected by conditional compilation, because not every dialect uses it
MasterDuke brrt: off-topic, but is there some reason github.com/MoarVM/MoarVM/tree/jit-moar-ops wasn't merged? 16:19
also, is this comment still correct? github.com/MoarVM/MoarVM/blob/mast...#L236-L240
brrt MasterDuke: ehm 16:20
yes
that's still correct
I'm working on that
But, it would've broken the nativecall-jit thing that nine++ has developed 16:21
which would've meant a slowdown
I didn't want to push that, but fixing it completely needed floating point support 16:22
and here I am :-)
MasterDuke ah, cool
brrt hmm 16:23
I"m not sure about jit-moar-ops
hmm 16:39
no, I just have no idea
Geth MoarVM/jit-expr-optimizer: 4 commits pushed by (Bart Wiegmans)++ 16:42
brrt at the risk of starting yet *another* branch....
MasterDuke is it easy to know when an SC's root_objects won't have anything more added to it? 16:44
brrt not that I know of 16:45
Geth MoarVM/expr-jit-invoke: 132e980222 | (Bart Wiegmans)++ | 4 files
[JIT] Implement invoke sequence in expr JIT
 16:50
MoarVM/expr-jit-invoke: 4dd9cd3e24 | (Bart Wiegmans)++ | src/jit/graph.c
[JIT] Eliminate special consume_invoke opcode

Except for nativecall_invoke, we no longer need this part of the legacy JIT.
MoarVM/expr-jit-invoke: 695d5e04d5 | (Bart Wiegmans)++ | 7 files
[JIT] WIP on nativeinvoke in the expression JIT
MoarVM/jit-expr-float: 9 commits pushed by (Bart Wiegmans)++ 16:58
MasterDuke brrt: another random question, could settypecache and setmethcache be jitted? 16:59
brrt I don't know what these do
I don't see why not 17:00
It'd be kind of a pain in the expression JIT now because it can't express loops
MasterDuke yeah, looks like their implementation would have to (at least partially) be moved out into a function 17:01
17:03 zakharyas left 17:11 brrt left 17:37 zakharyas joined 17:38 p6bannerbot sets mode: +v zakharyas
robertle can anyone explain to me how moarvm decides when to do GC? does it depend on some sort of "memory pressure"? is moarvm aware of some sort of max heap size? like the -Xmx settings on jvm? 17:38
18:02 lucasb left
nine robertle: MoarVM starts a collection run when a thread's nursery runs out of space. The nursery's size is defined by MVM_NURSERY_SIZE in src/gc/collect.h and is currently set at 4 MiB. 18:10
robertle: the start of full collections is documented as "This is determined by looking at the amount of memory that has been promoted to generation 2 relative to the overall heap size, and possibly other factors (this has been tuned over time and will doubtless be tuned more; see the code)." 18:12
in docs/gc.markdown
timotimo nurseries also start smaller for threads other than the main thread 18:29
nine So I have worked on this repossession thing all weekend and this is what I have actually done about it in the end: build.opensuse.org/package/view_fi...f?expand=1 18:34
timotimo you just put them together in the same file? 18:38
nine yep :) That's a workaround that I could have done on Friday already.... Anyway I still want to fix this for real but I need a solution tomorrow and I'm running out of time. 18:39
timotimo understandable
nine Our new year's resolution at $work is to switch to Perl 6 for all new functionality instead of just using it at the fringes. Last Friday I gave a little presentation on how to do this with our Catalyst applications and that's where I discovered that the Inline::Perl5 package that's installed on our servers was horribly broken. 18:40
timotimo damn!! 18:41
horribly broken but not caught by tests
or something?
nine No, because it only appeared when the module is installed through CompUnit::Repository::Staging which is only used when creating rpm packages. Zef doesn't use it yet.
timotimo ah, ok 18:42
19:06 zakharyas left 19:12 brrt joined 19:13 p6bannerbot sets mode: +v brrt
robertle nine: will read up on that, but to me this sounds like there is no way to tell moar how much heap to use? "realtive to the 19:17
overall heap size" is a bit ambiguous in that
timotimo it'll just grow as it needs to
there's no "start at this" or "stop at this" parameter 19:18
robertle ok, thanks 19:20
timotimo do you want to implement it so that starting perl6 only takes 5 megabytes of ram? :) :) 19:21
19:25 zakharyas joined 19:26 p6bannerbot sets mode: +v zakharyas
brrt we'll need a bit more than a max-heap-size parameter 19:33
imho, heap size parameter tweaking is the worst bit of the java platform
btw, we don't support finalization, do we?
19:35 MasterDuke left
nine brrt: finalization? 19:58
lizmat
.oO( this is the end, my friend )
 20:03
brrt yes, as in, running a piece of code if an object is deleted 20:07
nine We do: submethod DESTROY 20:08
brrt :-(
oh well
nine Sucks to support it, sucks to not have it... 20:09
20:15 zakharyas left 20:25 domidumont left
brrt yeah 20:32
robertle I agree that the constant fiddling with heap settings is super annoying in java, but at the same time I find it hard to see how a runtime can make quality decisions on GC without knowing the cost of increasing the heap. if I have plenty of memory, I might want to GC less frequently 20:40
but I am not complaining by any means, I am trying to understand and steal some ideas for my own totally sub-standard GC :)
one thing I noted was that I can increase my heap easily if required, but it is really hard to give anything back to the OS. so what work I have done in the past can easily affect my GC decisions now, which seems weird too 20:42
brrt so the thing is 20:44
you can't
this is the signature weakness of the whole model
and it is *the* reason why java is, above all, an *application* language 20:45
robertle what do you mean with "you cann't"? give back ram? 20:46
brrt You can't determine what is the 'correct' amount of memory use, from the limited PoV of the VM
given that the VM is a user process among equals
robertle ah, I get it
yeah
hence the annoying tuning
brrt The thing that you can do is assume you own the whole machine 20:47
robertle which is really terrible 20:48
brrt indeed
so there's this meme, for lack of a better word, this concept among IT folks that java is an excellent systems langauge
And it's not
It has this dumb lazy loading stuff that is totally unsuitable for the job 20:49
it has a smalltalk-ish 'we own your box' idea, that is totally unsuitable as well
robertle I guess most java people have gotten used to allocating memory to their processes 20:50
brrt The alternative is actually letting the JVM own your box, but if you do that, now suddenly java's GC has to fight its thread model to GC however many GB your heap is
and oracle can pay PhD's to design any number of GC algorithms with ever more evil names 20:51
robertle right, that's the other part: if you collect less frequently you get more total throughput, but the individual GC pauses go up
brrt oh, yes
and what about cache locality during tracing
(i.e. there is none)
robertle right, but we do agree that collecting too often isn't good either of course. so what I am struggling is how to determine the right frequency. currently I collect every N objects, which is of course dumb 20:53
you are saying moar does collect every time it has allocated a certain fraction of the total heap
certainly better
java has this idea of memory pressure, that it allocates more frequently if it gets short on heap space 20:54
also: when do you extend your heap? I am thinking when the "net" usage of the heap has reached some level 20:55
brrt I don't know that precisely 20:57
Geth MoarVM: czurnieden++ created pull request #1034:
adapt bigint handling to new LibTomMath version		 20:59
21:16 patrickb left 21:30 brrt left 21:36 Kaiepi left 21:38 Kaiepi joined, p6bannerbot sets mode: +v Kaiepi 23:14 MasterDuke joined, p6bannerbot sets mode: +v MasterDuke, MasterDuke left, MasterDuke joined, herbert.freenode.net sets mode: +v MasterDuke, p6bannerbot sets mode: +v MasterDuke
MasterDuke jnthn: have you seen any of my questions the past day or so? timotimo++ helped me figure out the index()/speshresolve problem seems to be deopt related, but i'm now *way* over my head 23:16
jnthn MasterDuke: No, haven't really been about here much over the weekend 23:23
Will take a look tomorrow
Certainly too tired now to think about deopt things :)
MasterDuke cool, thanks 23:25

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!