Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:03 reportable6 joined 02:01 greppable6 left, nativecallable6 left, coverable6 left, shareable6 left, releasable6 left, bisectable6 left, reportable6 left, committable6 left, evalable6 left, linkable6 left, sourceable6 left, statisfiable6 left, bloatable6 left, quotable6 left, benchable6 left, notable6 left, unicodable6 left, tellable6 left, squashable6 left 02:02 notable6 joined, bisectable6 joined, reportable6 joined, committable6 joined 02:03 benchable6 joined, shareable6 joined 02:04 greppable6 joined, quotable6 joined, statisfiable6 joined, tellable6 joined 03:03 evalable6 joined 03:04 unicodable6 joined, sourceable6 joined, squashable6 joined 04:03 linkable6 joined 05:04 nativecallable6 joined 06:02 reportable6 left 06:03 releasable6 joined 06:05 reportable6 joined 07:03 bloatable6 joined, coverable6 joined
Nicholas good *, #moarvm 08:05
I'm no longer sure which spectests are "expected to fail" on new-disp (versus what might be problems from high load on a shared machine) 08:06
but t/spec/S02-types/num.rakudo.moar and t/spec/S02-types/range.t both seem to reliably end up failing with: No such method '&postcircumfix:<{ }>' for invocant of type 'List' 08:07
08:15 lizmat_ joined 08:16 TempIRCLogger__ joined 08:17 Geth left, lizmat_ left, Geth joined 08:18 lizmat joined
lizmat jnthnwrthngtn: fwiw, test-t feels a bit slower after the last new-disp commits 08:29
startup time seems to have gone down a bit, between .135 - .161 now for me 08:30
jnthnwrthngtn: one data point. restarting my local copy of the IRC log server with new-disp, I get: 08:37
Cannot unbox a P6opaque (MAST::VOID) type object 08:38
when doing "use RandomColor"
(during compilation)
09:07 brrt joined
Nicholas good *, brrt 09:11
brrt good * Nicholas 09:16
(70% of the reason for coming here is the habitual greetings :-)) 09:17
lizmat brrt Nicholas o/ 09:18
brrt ohai lizmat 09:21
jnthnwrthngtn Nicholas: I'm only aware of require.t causing bother 09:43
lizmat: Curious; the profile I did of it came out with a shorter time, better inline rate, etc. 09:44
lizmat I only checked wallclock time
jnthnwrthngtn I'd have kinda expected that to follow 09:45
Good you see a (slight) startup improvement also; I saw a few seconds of `make spectest`
*off 09:46
09:58 brrt left
Nicholas not quite "reliably", but at least mostly, and it seems to be MVM_SPESH_NODELAY=1 10:04
jnthnwrthngtn There will be a MoarVM talk at this year's VMIL: 2021.splashcon.org/home/vmil-2021#program 10:06
urgh, on my office machine I can get lots of spectest failure with nodelay + blocking 10:08
Wonder what's caused that.
lizmat weekly: 2021.splashcon.org/details/vmil-20...g-language 10:09
notable6 lizmat, Noted! (weekly)
MasterDuke that graalvm talk looks interesting also 10:11
jnthnwrthngtn OK, can confirm it's the HEAD commit on new-disp that is to blame but...how... 10:12
Hm, and also, it isn't happening every time. oh, that may be hash randomization... 10:13
MasterDuke oh, little pricier than i'd prefer though...
Nicholas sorry, yes, already had assumed that randomisation
that randomisation was why it wasn't 100% reliably unhappy 10:14
jnthnwrthngtn Hm, and also, it isn't happening every time. oh, that may be hash randomization...
oops
this isn't the terminal I meant to up-arrow + enter in :D
Yeah, it is reliable with hash randomization turned off 10:17
Geth MoarVM/new-disp: bc95088a24 | (Jonathan Worthington)++ | src/spesh/inline.c
Tweak sp_guardhll deopt index during inlining
 10:40
jnthnwrthngtn That fixes at least the num.t case and quite possibly the rest of them 10:41
Nicholas building the whole thing to find out...
jnthnwrthngtn So, what was I meant to do next... 10:42
Nicholas coffee!
jnthnwrthngtn Already just got a fresh cup :)
Nicholas well you crossed out all the blockers on github.com/rakudo/rakudo/wiki/Raku...isp-branch 10:43
so it' snot obvious, but IIRC there were about 3 modules that did things 10:44
and lizmat reported something else this morning
jnthnwrthngtn I suspect what lizmat reported is going to be the thing I just fixed
MasterDuke it might be worth actually making the pull request about now so we get the CI runs
jnthnwrthngtn Will we, though, or will it be confused 'cus it needs simultaneous merge over all 3 repos? 10:45
Nicholas I suspect the latter. Also, what does the CI do that we can't (collectively) do manually? and have been doing already manually
MasterDuke arg, right. we'll at least get confirmation that moarvm builds, but the nqp and rakudo parts won't work 10:46
jnthnwrthngtn Goodness, a blocking+nodelay spectest on this machien is not so fast as at home...
.oO( this machien runs like a dog... )
Yeah, the fix seems to deal with all the spectest fallout 10:55
Currently adding a megamorphic handler for nqp::can/nqp::findmethod/nqp::tryfindmethod, since we regularly get blowups in mergesubstates 10:56
(When compiling anything non-trivial) 10:57
lizmat jnthnwrthngtn: alas, the MAST::VOID problem has not gone away: gist.github.com/lizmat/2c3336a3398...776b082051 11:19
dogbert17 lizmat, does test-t run faster now 11:23
jnthnwrthngtn lizmat: It's possible 8cb07b51 in Rakudo is to blame 11:24
lizmat dogbert17: no noticeable difference for me
11:25 linkable6 left
lizmat jnthnwrthngtn: want me to revert that locally and see if that fixes it ? 11:25
dogbert17 not even after the fix jnthn did 30 mins ago?
lizmat no :-(
dogbert17 darn 11:26
lizmat rebuilding with: $ perl Configure.pl --force-rebuild --gen-moar=new-disp --gen-nqp=new-disp --make-install;
that should give me the latest, no ?
11:27 linkable6 joined
dogbert17 just ran a stresstest with MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 and a small nursery (32k). The only failure was t/spec/S11-modules/require.t 11:27
I guess nine++ is looking at that one 11:28
jnthnwrthngtn lizmat: Yes, please
lizmat ok, rebuilding :-)
dogbert17 Nicholas: should one be worried if running a raku script under valgrind, with new-disp, looks ok but throws valgrind errors when MoarVM is built with --no-optimize? 11:35
11:36 brrt joined
dogbert17 hello brrt 11:36
brrt hello dogbert17 11:37
lizmat jnthnwrthngtn: indeed,. reverting 8cb07b51 fixes the MAST::VOID issue 11:38
11:39 linkable6 left 11:42 linkable6 joined
jnthnwrthngtn OK, hmm 11:42
Intersting all of spectest doesn't manage to tickle that 11:43
Geth MoarVM/new-disp: 3a961fa96f | (Jonathan Worthington)++ | src/disp/boot.c
Don't always guard on type/name in lang-find-meth

Otherwise, it's not possible for a language's dispatcher to handle megamorphic cases.		 11:56
MoarVM/new-disp: e1acd103fc | (Jonathan Worthington)++ | src/disp/program.c
Relax existence requirement on DP lookup tables

We can install a lookup table at a callsite in order to handle megamorphic dispatches. However, the mechanism that existed so far always made lack of entry in the table a reason for the dispatch to be rejected. This is not useful for writing dispatchers where lack of an entry is acceptable. Dispatchers that want the current semantics can obtain them simply by adding a guard.
jnthnwrthngtn That and the matching NQP change might just get a bit more off stage parse 12:00
12:02 reportable6 left
jnthnwrthngtn lizmat: Hopefully the change just pushed to Rakudo HEAD will fix the issue you ran into 12:18
well, rakudo/new-disp
lizmat ok, will do a rebuild :) 12:19
Nicholas "this week" is so last week. It's not even "this morning" any more. It's an acceleraptor
lizmat jnthnwrthngtn: confirmed this fixed the issue 12:31
MAST::VOID issue
jnthnwrthngtn Yay 12:38
Geth MoarVM/new-disp: 5e982bfc98 | (Jonathan Worthington)++ | src/disp/syscall.c
Add syscall to check if code is marked as stub
 12:49
jnthnwrthngtn Nice, exposed more constants in raku-invoke (where all invocations bottom out) and it gives some improvements across a bunch of microbenchmarks 12:56
Did this in NQP's dispatchers a while back and forgot to add it into Rakudo's ones 12:57
Pushed it to Rakudo. Might help test-t. 13:02
.tell sena_kun For Test::Base I've opened github.com/tokuhirom/p6-Test-Base/pull/2 13:09
tellable6 jnthnwrthngtn, I'll pass your message to sena_kun
lizmat 1.419 / .785 13:12
jnthnwrthngtn What were yesterday's numbers?
lizmat 1.451
.766 13:13
so, feels a bit faster for the non-race case, but definitely in noise territory
afk for a few hours&
Nicholas jnthnwrthngtn: OK, on 8cb07b51f14e1e0702bc51d4d4742476e8aca666 (which is yesterday) t/spec/S11-modules/require.t is the only spectest failure 13:14
(as in, a build that was started about an hour ago) 13:15
so your fix this morning got us back to your single known failure 13:16
jnthnwrthngtn OK, good
.oO( why is it my failure? :D )
Guess I should deal with DateTime::Timezones at last, before painting any more go faster stripes on things... 13:17
Nicholas as in "the single failure known to you". Pedant! :-)
the knowledge is yours. It might be possible to palm the failure off on someone else. 13:18
jnthnwrthngtn nine++ was looking at it; I struggled to repro it in a way that I got useful info 13:19
13:23 brrt left
Nicholas jnthnwrthngtn: not a massively scientific number, but "this morning" under ASAN Stage parse was about 400s. It's now 375s. Meaning (I think) that more got JITted 13:38
but the numbers have flapped up and down
ASAN and NODELAY make it all meaningless. And then there's hash randomisation on top 13:39
jnthnwrthngtn Hm, seems the DateTime::Timezones problem is a different one than I first thought... 13:44
dogbert17 jnthnwrthngtn: is that good or bad? 13:54
jnthnwrthngtn Bad in that I think the thing I originally thought it was will be a problem at some point too 14:09
I thought the case was going to be wrapped multi candidate that does callwith, but it's actually wrapped proto with callwith 14:11
They need distinct handling 14:12
14:23 evalable6 left, linkable6 left 14:25 linkable6 joined, evalable6 joined
jnthnwrthngtn Hm, seems I have a fix for DateTime::Timezones 14:28
But it breaks at least one spectest :(
14:59 patrickb joined
jnthnwrthngtn Figured it out. \o/ 14:59
dogbert17 jnthnwrthngtn++ 15:00
jnthnwrthngtn So that's one of our remaining ecosystem failures sent a PR, and one fixed. 15:02
timo \o/ 15:05
15:06 reportable6 joined
Nicholas and one last one to put on the wiki? 15:10
\o/
-bash: o/: No such file or directory
that appeared to be the wrong window :-) 15:11
jnthnwrthngtn Nicholas: Yeah, I guess so. Though we've put in so many opts and other tweaks we might have "won" some more ecosystem problems 15:24
15:28 patrickb left
[Coke] There's no blockers ATM; while that list may grown and shrink before next release, anything stopping us from doing the merge/rename? 15:29
jnthnwrthngtn [Coke]: I'd like to see another round of Blin results; a lot happened since the last one 15:33
My latest Rakudo push is the last place I know we were not exposing an opportunity to spesh that it had before. 15:45
15:49 patrickb joined
[Coke] Sure; let's do blin, get the results in the wiki. good plan 15:56
16:03 patrickb left 16:05 brrt joined
Geth MoarVM/new-disp: 7ac7b89741 | (Timo Paulssen)++ | src/spesh/worker.c
keep "skip" value around between loop iterations

fixes "skip:0" lines showing up everywhere in the log		 16:11
nine I asked why caller_deopt_idx wasn't set when it's the mechanism that's supposed to protect us from the "we have moved into/out of an inline since the MVMContext was taken" situation. 16:16
Well apparently at the time the MVMContext was created and the call stack was snapshotted, the REQUIRE_IMPORT callframe didn't have a spesh candidate. But later, when we apply the traversals it suddenly has one. 16:17
jnthnwrthngtn Hmm. OSR? 16:18
nine Looks like. Disabling OSR gets rid of the failure
And there are 2 osrpoints in the function's bytecode 16:20
timo do we poison osr when a mvmcontext was taken or do we somehow fix up mvmcontext when an osr has happened in the mean time?
i guess we can probably have a "which candidate was it at the time" in the snapshotted call stack 16:21
lizmat latest numbers" 1.389 / .781 16:24
jnthnwrthngtn I don't think we prevent OSR in such a situation, no 16:26
lizmat: ooh, does that mean at least for the non-race case, we're within startup time difference of master?
nine Preventing OSR for a frame that has caller_info_needed set in its extra is easy and works. Is it good enough as a fix? 16:27
timo does it prevent 99% of all osrs?
lizmat jnthnwrthngtn: I'd say, yes
nine caller_info_needed is set by exceptions and MVMContext 16:28
16:32 [Coke] left
timo i see a tiny opportunity to improve returning native values; in this example we have a NumLexRef and we go via the container-fallback code for the raku-rv-decont dispatcher. we don't get "isrwcont" for the known type of NumLexRef, so the code would get one less branch 16:33
but also, the raku-rv-decont dispatcher could learn about the *LexRef and other native refs
jnthnwrthngtn nine: I think it will be, yes 16:34
nine Doesn't seem to affect stage parse time
jnthnwrthngtn I think I just spotted one more spesh regression
We used to log return types and insert guards
We no longer do
lizmat timo: having native values not being containerized would be cool 16:35
timo that's not a thing i can promise
lizmat timo: it's basically blocking using natives for hot code
timo when it's a LexRef, that is definitely a rw container, meaning we do have to re-containerize
lizmat if any subs / blocks are involved
timo this in particular is multi method wrap(numarray:D:) 16:40
Geth MoarVM/new-disp: a7c0cc8d2b | (Stefan Seifert)++ | src/spesh/osr.c
Fix frame walker getting confused when OSR outdates positions

When an MVMContext is created, we snapshot the current callstack, i.e. record the current positions within the frames in their extras. These positions are relative to the currently executing bytecode. OSR however replaces this bytecode with specialized one, invalidating the recorded positions. This can lead to the frame walker making wrong decisions with regards to whether we were ... (5 more lines)		 16:46
timo sum isn't huge, but can't be inlined into its caller in this case because it's got getlexref for $sum 16:48
Nicholas oh my, "I" had not finished the previous build yet... 17:15
Geth MoarVM/new-disp: 508299f952 | (Jonathan Worthington)++ | src/jit/graph.c
JIT of iscoderef
 17:26
MoarVM/try-return-type-guards: 7ae62af9ed | (Jonathan Worthington)++ | 6 files
[WIP] Bring back return type guards

We log the types that things return, with the intention of using those to add speculative guards that might be used during specialization. While this does speed some things up, it also causes some huge slowdowns, so needs further investigation.		 17:28
jnthnwrthngtn That's a bit of a mystery. I see some benchmarks half in time with that, but others double in time; test-t tanks too. 17:29
I thought it was because of overlapping type data with the following op, mitigated that, and it didn't help 17:30
I thought it was because of overlapping type data with the following op, mitigated that, and it didn't help 17:31
home time & 17:35
17:37 linkable6 left, evalable6 left 17:54 squashable6 left 17:56 squashable6 joined 18:02 reportable6 left 18:07 brrt left 18:39 linkable6 joined
Nicholas So t/spec/S02-types/isDEPRECATED.rakudo.moar is (mostly) failing for me on new-disp. "not ok 4". 18:45
And on master. "not ok 13" 18:46
"not ok 4" is expected "t/spec/S02-types/isDEPRECATED.rakudo.moar, lines 31,32" got "t/spec/S02-types/isDEPRECATED.rakudo.moar, lines 18,31"
little of this makes sense to me.
lizmat those are tests to see if the "is DEPRECATED" trait reports the correct line numbers 18:47
for the currently active deprecations
in core
jnthnwrthngtn Just profiled test-t with the try-return-type-guards branch and it has a huge number of deopts 18:49
Nicholas OK, the "mostly" seems to be that if I delete all of lib/.precomp it passes, and then on the re-run(s) it fails 18:50
lizmat Nicholas: yeah, I think we still have some gremlins in the precomp selection stuff 19:00
jnthnwrthngtn OK, I figured out the problem with the return type logging 19:08
Turns out that values from the recording of the dispatch program were also getting logged
These of course have nothing to do with the actual call 19:09
And in some cases where the result of the dispatch is not a call, but a value, they produce utterly bogus stats
That said, we probably shouldn't be guarding on such weak type counts.
19:48 squashable6 left 19:50 squashable6 joined 20:03 reportable6 joined
MasterDuke it looks like we can't jit anything that involves (un)locking a mutex (assuming it's not all hidden behind a function call)? 20:18
sort of related question, is there any real downside to jitting all ops? i.e., would it be better to not jit ops that are very infrequently called, and (almost) never in hot code? or is there an extra benefit to having everything jitted so you're not spending time in the interpreter? 20:42
lizmat it takes effort to jit? 20:53
jnthnwrthngtn MasterDuke: Depends what you mean; having ops JITted unblocks JIT of those things around them 20:54
So even if they aren't that hot, it can be worth it.
It's not always worth trying to JIT things better than "call a function" though (as it can blow up the generated code size) 20:55
MasterDuke so i'm looking at the spesh log of compiling CORE.c.setting. there are a bunch of unjitted ops that only prevent 1-<small number> of functions being jitted
jnthnwrthngtn Then it depends how common those are 20:56
MasterDuke i.e., 'loadlib' prevents 'name' from being jitted
jnthnwrthngtn I addes iscoderef earlier today and that turns out to be important because one of the dispatchers uses it
MasterDuke if 'name' is called only infrequently, might it still be worth jitting 'loadlib'? 20:57
i also wonder how easily it would be to measure jitting it vs not. do a whole bunch of these ops and then see if there's a measurable difference, either good or bad? 20:58
jnthnwrthngtn It must be at least somewhat frequent because it's getting specialized 20:59
Anything that's called < 100 times or so won't be
MasterDuke i generate two metrics. ops -> the functions that they prevent from being jitted. and functions -> the ops that prevent them from being jitted 21:00
so two categories of ops seem most helpful to jit, those where there are a large number of functions they block, and those where the functions they block are especially hot 21:02
the first is easy to spot. the second requires more inside knowledge 21:03
gist.github.com/MasterDuke17/d3d42...80b1ddcab3 shows the op->function data 21:05
something like 'setdebugtypename' is just a function call, so you think it would most-likely be good to jit it? 21:08
21:13 [Coke] joined
timo setdebugtypename could one day™ become a syscall and jitting to a c function could be for free then 21:19
jnthnwrthngtn I figure a bunch of ops today should eventually become syscalls 21:24
wow, how'd this happen... 21:49
(gdb) p STABLE(coderef_facts->value.o)->debug_name
$4 = 0x555555725540 "NQPMu"
(gdb) p STABLE(coderef_facts->type)->debug_name
$5 = 0x5555555efd10 "BOOTCode"
MasterDuke i assume this is entirely coincidental to our just talking about setdebugname? 21:50
jnthnwrthngtn Yes :) 21:52
MasterDuke ha
jnthnwrthngtn Figured this'd be the perfect time for rr, but...it crashes :/ 21:57
timo if your program crashes, rr will also report a crash
so it might just look like it crashed
jnthnwrthngtn No, rr replay itself crashed, turns out that I needed to run the zen workaround script 21:59
timo oops ok
jnthnwrthngtn Seems whatever it does only sticks until a reboot 22:09
MasterDuke yeah, you can do a kernel module or boot params instead if you want something more permanent 22:18
timo React Router is being mean to me :| 22:27
Geth MoarVM/new-disp: 9c1416d07e | (Jonathan Worthington)++ | src/spesh/codegen.c
Add sp_guardhll to codegen debug
 22:29
MoarVM/new-disp: 005061d0dc | (Jonathan Worthington)++ | src/spesh/inline.c
Correctly update guard deopt indice on inline

If there were to be multiple inline annotations, a pre and a post one, then we should respect the one the op has encoded in it.
MoarVM/new-disp: 229d37e80d | (Jonathan Worthington)++ | src/spesh/usages.c
Consider all kinds of deopt when adding usages
MasterDuke timo: i have a feeling that for all raku is a large language, javascript + frameworks + ecosystem is still *vastly* larger than raku + frameworks + ecosystem 22:33
Geth MoarVM/new-disp: b9204f6b35 | (Jonathan Worthington)++ | src/spesh/manipulate.c
Don't copy facts in version split

It's possible in odd situations (created by NODELAY, where the stats are liable to be dubious at best) to end up with facts being on a virtual regsister that aren't a subset of those added by the guard. In that case, we could end up with bogus combinations of the two. Just don't copy them; it seems there's little to no win anyway.		 22:36
timo wouldn't surprise me, javascript has been growing at a strong pace for the last 500 years now
Geth MoarVM/new-disp: 36f4d79426 | (Jonathan Worthington)++ | 6 files
Reinstate adding speculative return type guards

We were logging return types, however due to changes to how dispatch works, were completely ignoring them in the specializer. Change that. We need to do things a little differently, because dispatch is a varargs op, so we can't just calculate the offset before it trivially. Instead, use a post-log annotation. However, that can collide with a log ... (12 more lines)
jnthnwrthngtn For the first time, I just observed a stage parse on new-disp lower than that of the one I last measured on master.
MasterDuke ooo, going to give it a try myself then 22:38
22:38 evalable6 joined
timo sweet 22:38
MasterDuke 44.5 here. don't remember what my last master time was, somewhere around 42-44 i think 22:42
Geth MoarVM/new-disp: 1a494f5788 | (Jonathan Worthington)++ | 3 files
Don't double-check REPR in sp_guardsf

We will always have either knowledge or guards to make sure it is.		 23:14
[Coke] updating to latest os x (was a major rev back), will report out once my tooling is back in place. 23:41

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!