Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:03 coleman-x left, coleman-x joined 00:04 reportable6 joined 00:22 evalable6 joined 00:52 ggoebel left 01:24 linkable6 joined 05:18 kjp_ is now known as kjp 06:02 reportable6 left 06:06 MasterDuke left, bartolin joined, japhb_ joined 06:10 jdv_ joined 06:14 timo left, lizmat left, MasterDuke left, harrow left, japhb left, ilogger2 left, Util left, bartolin_ left, raydiak left, jdv left 06:15 timo joined 06:16 lizmat joined, harrow joined 06:17 Util joined 06:20 raydiak joined 06:25 ilogger2 joined
Nicholas good *, #moarvm 06:37
unless you're big endian, in which case your morning is still a bit LTA 06:38
06:38 patrickb joined 06:50 harrow left 06:53 harrow joined 07:03 reportable6 joined 07:08 ggoebel joined 07:16 Kaipi joined, ggoebel_ joined 07:18 shareable6_ joined 07:21 shareable6 left, Kaiepi left, ggoebel left, tellable6 left, tellable6 joined 07:22 childlikempress joined, moon-child left, childlikempress is now known as moon-child 07:25 committable6_ joined, notable6_ joined 07:27 notable6 left, committable6 left 07:31 samcv_ joined 07:34 samcv left 08:17 jdv_ left, jdv joined
Nicholas nine: if "all" I am doing is defining MVM_LOG_DEOPTS 1, how do I know if a deopt was "recent"? 08:22
I can see what the most recent logged thing is before my breakpoint on MVM_exception_throw_adhoc is hit, but I don't know what that then tells me. 08:23
08:29 bloatable6 left, bisectable6 left 08:30 bloatable6 joined, bisectable6 joined 08:40 Kaipi left 08:42 Kaiepi joined, reportable6 left 08:43 reportable6 joined
nine 08:48
moon-child 08:49
nine Nicholas: what does it tell you?
Nicholas Deopt one requested by interpreter in frame 'raku-multi-plan' (cuid '198') Will deopt 4128 -> 4344 Completed deopt_one in 'raku-multi-plan' (cuid '198') 08:51
Breakpoint 1, MVM_exception_throw_adhoc (tc=0x10030b90,
(but in 5 lines, not 2)
nine That's not the same frame as your bisection result. Are there any deopts in 'val' at all? 08:52
May be worth a try to verify the jit-bisect result manually. Sometimes it's a bit off unfortunately :/ Disabling hash randomization may help getting a stable result 08:53
Nicholas can't see on in val
and I think that this is the same output as last night
and it's a different ppc64 machine.
can't see one in val
usign the first ppc64 machine to grind forwards trying to find the last good rakduo/nqp/MoarVM combo 08:54
commit 42b52ec9e392b00b9114b1c213ee4ae14ca4d789 was good
08:55 linkable6 left 09:05 ggoebel__ joined 09:07 jdv left, jdv joined 09:12 jdv_ joined 09:14 ggoebel_ left, jdv left, moon-child left, moon-child joined
jnthnwrthngtn Nicholas: I always disable hash randomization when doing spesh/JIT bissects, because it reliably makes them useless 09:34
(In fact, I'd probably have been better disabling it throughout the new-disp work and reenabling it before merge... :)) 09:36
nine I wonder if it'd have a measurable cost to make it possible to disable hash randomization via an environment variable? 09:41
jnthnwrthngtn I think it'd be tested on some hot paths, but it maybe possible to arrange things so not 09:42
lizmat it would be nice to have it environment variable settable 09:44
nine It'd be an MVM_LIKELY(tc->instance->hash_randomize) per hash we construct 09:46
lizmat that feels expensive?
jnthnwrthngtn Hmm, despite the vast improvement there are still 13754 attempts to transition a full dispatch cache in the p6-mqtt thing. 09:47
Nicholas it also needs to be a check on that (or similar) at the start of every string we feed into siphash
jnthnwrthngtn That'd be the most costly place, I guess? 09:49
nine Nicholas: does it? I don't see any checks for MVM_HASH_RANDOMIZE there?
Nicholas actually, not. but it's the setup of tc->instance->hashSecrets[0] and tc->instance->hashSecrets[1] that also needs to be changed to a fixed value (ie 0) if hash randomsiation is disabled 09:52
else one gets different results from MVM_string_hash_code for each run, and that will make different hash ordering
(too)
nine Yes. I didn't mention that because it's a one time thng 09:53
Nicholas true. I was confused. But it's still "don't forget..."
jnthnwrthngtn Ah, then maybe not too terribly costly 09:56
Should we worry about folks setting the env var instead of fixing their busted code/tests? :) 10:03
timo could require -DDEBUG for that perhaps
then it will be available mostly to devs, but normal users don't get the performance hit 10:04
dogbert17 m: &grep does role :: { method CALL-ME (|) { say "Got a call"; }} ; grep(Str) # correct? GH #4543 10:11
camelia ( no output )
timo m: &grep does role :: { method CALL-ME (|) { say "Got a call"; }} ; grep.(Str) 10:14
camelia 5===SORRY!5=== Error while compiling <tmp>
Calling grep() will never work with signature of the proto (Mu, |)
at <tmp>:1
------> 3od CALL-ME (|) { say "Got a call"; }} ; 7ā5grep.(Str)
timo m: &grep does role :: { method CALL-ME (|) { say "Got a call"; }} ; &grep.(Str)
camelia ( no output )
timo feels a little strange to put a CALL-ME on a sub, though that's kind of what nativecall does 10:15
the normal way to do this would be with wrap though, right?
Geth MoarVM/hash_randomization_environment_variable: c6c7041781 | (Stefan Seifert)++ | 6 files
Support disabling hash randomization via environment variable

This makes it easier to debug and get useful results from e.g. jit-bisect.pl.
nine timo: what NativeCall used to do
dogbert17 c: 2021.09 &grep does role :: { method CALL-ME (|) { say "Got a call"; }} ; grep(Str)
committable6_ dogbert17, Ā¦2021.09: Ā«Got a callā¤Ā»
jnthnwrthngtn Even if we reinstate paying attention to the mixed-in CALL-ME, it's not going to impact any callsites already bound to an outcome pre-mixin. 10:24
The immutability of the lexpad post-CHECK time really should be considered to extend to Routines installed into it at compile-time also.
However, we can't just forbid `does` on a Routine because that's a perfectly fine thing for a trait to be doing. 10:25
lizmat but does are at compile time, generally, no? 10:27
*those
Geth MoarVM: 1420640ab2 | (Jonathan Worthington)++ | 3 files
Add a way to request non-installation of a DP

In some cases, we may know that we are only recording a new dispatch program because of a transient situation. For example, a megamorphic by type method dispatcher may end up in record mode because the class in question didn't have a cached flat method table yet. Once that situation is resolved, an already installed dispatch program could start to match. Thus provide a way for a dispatcher to indicate that it's not worth the installation of the current dispatch program, so we can hopefully avoid one cause of piling up a mass of programs at the callsite beyond the point we realize it's megamorphic.		 10:29
jnthnwrthngtn lizmat: Yes, they are, and it's possible that the issue was golfed from such a situation too. 10:30
Hm, that plus a matching NQP commit are not quite so helpful as hoped... 10:32
lunch, bbl
lizmat jnthnwrthngtn: yeah, basicallt from 8.0 to 7.9 for MQTT::Client test 10:36
*basically
ah, missed the NQP one: with the NQP one, it's down to 7.8 for me 10:43
10:48 Altai-man joined
Nicholas OK, first (of many?) big endian bug appears in MoarVM between 720b1aa169de2fb508f7ae80e095b89fad30af4b and 649c9ccad21debf80fec48025b824bad07464785 10:52
actually, end is tigher than that. It's present at e9be25809ccfdf13b9db4beeba2a4f9cf0feb535 (which changes which op is reported as NYI) 10:53
nine So, just 12K lines of diff to comb :D
Nicholas I'm trying to get down to 1 commit 10:54
nine Lets hope it will be the actual commit introducing the broken code, not just a commit unlocking some previously broken optimization 10:56
Though even the latter can point us in the right direction
Nicholas even if it's the latter, it reduces the number of thigns to look for
remember, I don't udnerstand spesh. but I do understand C :-)
10:56 linkable6 joined
Nicholas OK, it's wierder still. It's commit edb9e5bc04c1232799c378779681ba9a4096a898 10:58
edb9e5bc04c1232799c378779681ba9a4096a898 / 27d0b9ff6e0e86bf8763daef7927a8216f6a145b / b21bf9a80e380b17f72cd34b63d40552b8c79ed3 is bad (gets a NYI) 10:59
10:59 linkable6 left
Nicholas edb9e5bc04c1232799c378779681ba9a4096a898^ / 27d0b9ff6e0e86bf8763daef7927a8216f6a145b / b21bf9a80e380b17f72cd34b63d40552b8c79ed3 is good 10:59
11:00 linkable6 joined
nine Huh...can't claim that the error is blatantly obvious to me. But then, if it were I'd probably have avoided it in the first place 11:02
Nicholas prior to that commit, only 1 line does new_operands[3].lit_ui32 = deopt_index;
which size *should* a deopt_index be?
nine why deopt_index? 11:04
Nicholas it's assigned to lit_ui32
I'm suspicious about alignment in unions
nine The commit does new_operands[2].lit_ui32 = MVM_disp_inline_cache_get_slot(...). And the latter does return an MVMuint32 11:05
And there are 4 other places that do exactly the same
Nicholas but only 1 is uncommented? 11:06
and I don't know if it is reachable
nine No? optimize_getlexstatic does exactly the same 11:07
sp_assertparamcheck's arguments are defined in oplist as r(int64) sslot uint32 and write_instructions in codegen.c reads ins->operands[i].lit_ui32 for MVM_operand_uint32 parameters. So assigning to lit_ui32 should be safe, shouldn't it? 11:13
11:13 ggoebel__ left
Nicholas well, I believe so 11:13
nine Nicholas: can it really be something as basic as GET_UI32 in interp.c? 11:19
Nicholas I really dobut it 11:20
new strangeness - I stuck a breakpoint on sp_assertparamcheck and ok is 1 both times
11:21 ggoebel__ joined
timo turned off jit before? 11:21
Nicholas timo: this is ppc64
timo ah, of course
you're debugging endianness issues
Nicholas JIT is "NYI" :-)
timo SMOP, surely 11:22
Nicholas FSA_SIZE_DEBUG and valigrind find an invalid write at interp.c:248 11:33
OP(extend_i16): 11:34
GET_REG(cur_op, 0).i64 = (MVMint64)GET_REG(cur_op, 2).i16;
before this spesh problem.
so, the real bug is earlier...
timo huh, is that not targeting a 64bit register?
Nicholas unavoidable lunch& 11:36
timo non-maskable interlunch 11:37
11:38 ggoebel__ left 12:03 reportable6 left, reportable6 joined 12:15 rypervenche left 12:29 japhb_ is now known as japhb, rypervenche joined 13:25 [Coke] is now known as CokeZero, CokeZero is now known as [Coke]
jnthnwrthngtn lizmat: Yeah, it's something. Also it's now pretty close to what it was before new-disp for me. 13:33
(A bit slower, but not by that much.)
nine jnthnwrthngtn: looks like Inline::Perl5's slow down doesn't have anything to do with the method cache or find_best_dispatchee no longer existing. Looks more like optimized NativeCall sub bodies are no longer used 13:35
Have to confirm the latter though. Profiler's too broken to tell from that
timo :o 13:36
jnthnwrthngtn Profiler's broken? In a way specific to nativecall, or? 13:37
(I've used it quite a bit recently)
nine Well for one running with --profile is super slow compared to before. I also don't get a call graph anymore 13:38
[Coke] build seems broken on windows again. Might be me trying to force a build of all 3 bits from one config line.
nine Have to change trains. May be a bit
jnthnwrthngtn nine: Odd, I was seeing much smaller profiler output
(For the things I tried)
Including finally being able to do CORE.setting
So I guess there's something different about this case 13:39
nine testing CSV's csp-ip5xs.pl 13:45
profile time seems to grow exponentially with the number of rows
[Coke] seeing many more warnings in moarvm build today, weird. 13:46
nine 16K rows take half a minute, 100K rows take noeth of half an hour 13:47
[Coke] (wonder if I accidentally deleted some lines yesterday.)
jnthnwrthngtn o.O 13:48
I'd guess perf would show up something of interest
Nicholas I'm really strugglign with this big endian stuff. 14:00
[Coke] updated compiler warnings: gist.github.com/coke/4626a2760c314...794c55f2ac
Nicholas In that, I end up with valgrind reporting garbage reads (or writes) with spesh enabled 14:01
but from interp.c and non spesh ops
jnthnwrthngtn Normal ops are executed in spesh code
Garbage as in uninitialized or out of bounds? 14:02
Nicholas but if I build the same version on x86_64 with (the same) paranoia, all is happy
jnthnwrthngtn oh, writes implies the latter
Nicholas ==1619710== Address 0x119f62d8 is not stack'd, malloc'd or (recently) free'd
==1619710== Invalid read of size 8
jnthnwrthngtn Can you breakpoint it and get the register index it's trying to read?
timo perhaps even `call MVM_dump_bytecode(tc)` 14:03
Nicholas no, I can't see how I can 14:04
valgrind reports different errors from a gdb run
so I'm not getting a SEGV when usign gdb
timo in theory, doesn't valgrind also expose a gdbserver interface? 14:06
can you also get it to complain with asan?
i think asan has an env var option that lets you abort() when such an access happens
jnthnwrthngtn This is odd, some inline caches reach full entries even though I don't think that should be possible any more... 14:11
Or rather, in a situation I think it shouldn't be possible 14:12
yeah, this is weird, it should be boring 14:17
A simple method call
With a huge variance in invocant type 14:18
oh 14:20
It's boot-boolify 14:22
It guards on type 14:23
nine Not kiddimg: they just told us that they have to reboot the train. The IT Crowd was far more visionary than I thought 14:39
[A[A 14:47
jnthnwrthngtn Seems there is a little benefit in running latest dispatch program first (in order to get the megamorphic handler added most recently) 14:51
Nicholas I didn't know that ASAN had "UNKNWON" in its vocab: ==1629517==The signal is caused by a UNKNOWN memory access. 15:11
lizmat cosmic ray?
neutrino? 15:13
WIMP? 15:14
Geth MoarVM: f452513c57 | (Jonathan Worthington)++ | src/disp/inline_cache.c
Attempt latest dispatch program first

This means that for megamorphic sites that install a program to handle everything, we'll use that ahead of the rest. Rationale:
  * The dispatch programs we stacked up before installing the megamorphic
   one are liable to regularly fail, and so less efficient to attempt
... (7 more lines)
MoarVM: c7237e9c83 | (Jonathan Worthington)++ | 2 files
Translate dispatch ops used at megamorphic sites

So that we will be able to at least JIT compile the HOW access and hash lookup.
lizmat seems to gain another .1 on MQTT-Client test 15:30
jnthnwrthngtn Yeah, that's the kind of magnitude I saw now 15:31
How does it compare on your machine to the pre-new-disp measurement? 15:32
lizmat I don't think I have a proper local one
jnthnwrthngtn ah, ok, I was asking 'cus I don't on this box either 15:33
But I think we're probably within 10% now
lizmat yeah, feels like 15:34
jnthnwrthngtn Given this is one of the worst cases for the new-disp model (explosions of types all over the place), that's not too bad. 15:35
boot-boolify handles the megamorphic stuff quite badly 15:36
Figuring out a solution for that would likely help, I'm just not too sure what it'll look like yet
Probably pushing some amount of it out of the VM and into userspace 15:37
lizmat well, that's what you've been doing a lot of lately anyway, right?
jnthnwrthngtn Yup :)
lizmat so that would be a good thing, no?
jnthnwrthngtn I just need to decide how far to go with it, is all 15:38
lizmat right.... and I guess more userland stuff, means more warmup time
jnthnwrthngtn Do we toss the boolification spec entirely in MoarVM and do a full dispatch solution, for example.
Yes, there's that
Anyway, 10% slowdown is much more manageable than factor of 7 slowdown. 15:40
lizmat indeed :) 15:41
although 10% speedup would be better :-) 16:06
[Coke] :P 16:07
lizmat yeah, I know :-) 16:08
[Coke] ok. on windows today, I am getting failed to load library 'dynext\perl6_ops_moar.dll' at gen\moar\BOOTSTRAP\v6c.nqp:1134 (<ephemeral file>:)
moarvm and nqp appear to have built and installed...
ugh. nevermind. 16:10
timo the perils of using Configure.pl to build all three parts? 16:11
[Coke] (typo in the PATH setup, so while the --prefix was pulling from the right spot, trying to run the installed nqp later was not great.
jnthnwrthngtn lizmat: I suspect if we find a way to cache the derived regex languages between EVALs we'll gain at least that.
[Coke] timo: of NOT using, in this case. writing a .bat file to help me force all 3 to build at master 16:12
master/HEAD
timo oh my 16:13
nine Really looks like NativeCall is not using the generated function bodies anymore. Except during BEGIN time (just to make it weird) 16:16
16:18 MasterDuke joined
nine No, it's not BEGIN vs. runtime either 16:20
[Coke] nope, fixed my typo, still can't build rakudo on head. wtf. 16:29
Geth MoarVM: 6cdb0846c5 | (Jonathan Worthington)++ | src/6model/reprs/ReentrantMutex.c
Fix segfault when GCing ReentrantMutex

We cannot clone a ReentrantMutex. In the event that we attempt to do so, an object is allocated, but then not initialized. That lack of initialization would lead to a NULL pointer dereference when we tried to GC the lock. (We don't need to take care of this anywhere else, because the clone never leaks out to userspace; an exception is thrown before it can.)		 16:31
timo oh wow
16:31 coleman-x left
nine jnthnwrthngtn: when _do_ we fail to initialize a mutex? And can we actually recover in that situation? 16:33
jnthnwrthngtn nine: When we allocate the memory (in the GC), then call `copy_to`, and it throws 16:35
The GC still needs to clear up the allocation
hah, I wrote a Rakudo test to cover it, then realized that if I push it it'll SEGV unless I do version bumps... 16:36
nine Looks like the generated bodies that do get used are created by the CHECK phaser NativeCall exports: github.com/rakudo/rakudo/blob/mast...kumod#L792 16:41
But that should really be all of them. Without the CHECK phaser we don't call any generated bodies at all. With it only some 16:46
I really can't find a system in this 17:08
17:09 linkable6 left
timo is what jnthn just committed to rakudo about nativecall? 17:09
ah, no it's about the other exaple from earlier today
nine It's about why Inline::Perl5 is now twice as slow 17:10
timo i meant mixing CALL-ME into a proto
nine Ah, sorry, misread your sentence. Sounds like that, yes 17:11
jnthnwrthngtn What I just committed fixes a reported regression 17:16
17:23 patrickb left 17:34 Altai-man left
jnthnwrthngtn Time to go home, rest, etc. 17:34
18:03 reportable6 left 18:10 linkable6 joined
lizmat ... 18:57
19:04 reportable6 joined
jnthnwrthngtn Goodness, that was a filling dinner... 20:42
nine: I've been pondering what NativeCall might look like if really done to take advantage of new-disp. My rough idea: 20:47
1. Add a new dispatch terminal (along with value, constant, invoke bytecode, etc.) which is to invoke an FFI function.
2. Can add guards, decont Scalar containers, and add unboxings if needed as part of the dispatch program, so there's not much conversion work left in the final thing.
3. It specializes into guards and an op like `sp_nativecall`, which is what it then used on platforms where we can to JIT the native call.
Note that this means that the location we make a native call can end up, in the JIT case, with the C call itself emitted, so we're not even dependent on inlining of anything, nor any QAST generation, etc.
MasterDuke so i'm not terribly surprised, since that one was trickier, but reverting the jitting of scsetobj in github.com/MoarVM/MoarVM/pull/1552 got it farther. now nqp build but there's an error building rakudo i've never seen before 20:53
`NMAKE : fatal error U1077: 'C:\Strawberry\perl\bin\perl.exe' : return code '0xff` 21:03
i guess this means (at least) two of the ops are incorrect. ugh, another round of reverting to find out which one... 21:04
timo well that's super odd 21:41
we use perl for the exprjit compiler don't we? is that the part that asplode?
does nmake output more info? maybe with --no-silent-build? 21:42
MasterDuke it's somewhere in stage parse, since stage start finished 22:06
dev.azure.com/MoarVM/MoarVM/_build...b312016e60
timo ah, that's the "run_early_build_rakudo.pl" or whatever it's called 22:09
and it quite possibly just forwards the exit status from the moarvm process itself
MasterDuke i'm not really regretting now making a PR right away and just adding each new commit to it 22:16
*now really
*not making
[Coke] can roll things back and do a PR, no? 22:18
MasterDuke eh, i'm kind of in the middle now 22:19
[Coke] fair enough 22:27

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!