Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:05 MasterDuke joined 00:40 MasterDuke left 02:04 frost left 03:28 frost joined 04:28 nativecallable6 left, coverable6 left, tellable6 left, quotable6 left, sourceable6 left, greppable6 left, unicodable6 left, evalable6 left, releasable6 left, reportable6 left, notable6 left, benchable6 left, bisectable6 left, committable6 left, squashable6 left, shareable6 left, linkable6 left, statisfiable6 left, bloatable6 left, unicodable6 joined, tellable6 joined, committable6 joined, notable6 joined, coverable6 joined, benchable6 joined 04:29 reportable6 joined, squashable6 joined 04:30 bloatable6 joined, shareable6 joined 04:31 linkable6 joined, quotable6 joined 05:18 SmokeMachine left 05:19 leont left 05:21 leont joined 05:28 evalable6 joined 05:29 bisectable6 joined 05:30 releasable6 joined, nativecallable6 joined 05:32 SmokeMachine joined 06:03 reportable6 left 06:06 reportable6 joined 07:30 statisfiable6 joined, greppable6 joined, sourceable6 joined 08:30 evalable6 left, linkable6 left 08:32 linkable6 joined 09:08 MasterDuke joined 09:15 frost left, frost joined 09:30 evalable6 joined
MasterDuke i just jit bisected my branch where jitting ctxcallerskipthunks causes problems (but i believe because it's uncovering something else), but i don't remember what the output tells me 09:41
JIT Broken Frame/BB: 1460 / 1
Nicholas good *, MasterDuke 09:45
I don't think I ever knew this stuff, so I can't help
other than "I can pretend to be a rubber duck" 09:46
MasterDuke "behold the (rubber) duck, he doth not cluck. he clucking lacks, he quacks" 09:47
nine First number might be cuid 09:57
Ah, no, it's the MVM_SPESH_LIMIT!
Err... third time is the charm. It's actually the MVM_JIT_EXPR_LAST_FRAME 09:58
Geth MoarVM/new-disp-nativecall-libffi: 7 commits pushed by (Stefan Seifert)++ 09:59
MasterDuke hm. if i run with `MVM_JIT_EXPR_LAST_FRAME=1459 MVM_SPESH_BLOCKING=1` i still get the error 10:00
Nicholas Geth: I don't think that that first like summary was quite accurate... 10:01
nine MasterDuke: try 1458 and it will succeed? 10:02
MasterDuke nope, had to go down to ~39
but i'm going to try again after disabling hash randomization 10:03
huh. i see it bisecting and saying 1458 and 1459 are ok, but if i run with those i get the error 10:06
for me it bisects to 29 10:08
10:28 frost left
MasterDuke that took an annoyingly long time to figure out. it's setting MVM_SPESH_NODELAY=1 by default. for some reason i thought that was something you could tell it to do, but not the default 10:37
and now it's fine with MVM_JIT_EXPR_LAST_FRAME=1459 10:38
10:38 frost joined
MasterDuke jit breakpoints, never heard of these before 10:43
Geth MoarVM: a3c17d0f5d | (Nicholas Clark)++ | src/6model/reprs/NativeCall.c
When NativeCall is libffi, don't leak ffi_arg_types during GC

body->ffi_arg_types is allocated with `MVM_malloc`, so needs a corresponding `MVM_free`.		 10:59
MoarVM: a2a82f990a | (Nicholas Clark)++ | src/6model/reprs/NativeCall.c
When NativeCall is libffi, copy `ffi_ret_type` and `ffi_arg_types`

This bug was only exposed as a side effect of refactoring NativeCall to use new dispatch.
MoarVM/new-disp-nativecall-libffi: 7 commits pushed by (Stefan Seifert)++ 11:14
11:38 linkable6 left, evalable6 left 11:40 evalable6 joined 12:03 reportable6 left 12:04 reportable6 joined
dogbert11 Have anyone seen this error before? 'Directive x not applicable for value of type Int' 12:16
MasterDuke nope 12:20
dogbert11 got it when running t/spec/S07-hyperrace/basics.t 12:21
MasterDuke i think lizmat said she's seen the occasional fail in that one too 12:23
dogbert11 test #33 to be more specific
lizmat good well be, but I have no immediate recollection of that particular one 12:34
*could 12:35
dogbert11 seems to happen quite seldom (unfortunately) 12:40
12:40 linkable6 joined
dogbert11 lizmat, MasterDuke: gist.github.com/dogbert17/aba76e56...405c9d94ce 12:41
lizmat that comes from deep in the bowels, no ? 12:42
dogbert11 yeah 12:43
lizmat could it actually be an error in the error reporting ? 12:44
cause I don't see a .fmt used in the code ?
MasterDuke github.com/Raku/roast/blob/master/...ics.t#L141 12:45
lizmat argh, I was looking at the comment of the preceding test that succeeded 12:54
guess I'm not really awak yet
*awake :-)
13:06 discord-raku-bot left
dogbert11 at least it isn't spesh related 13:45
13:56 discord-raku-bot joined
nine Oh darn... INITIALIZE-DYNAMIC isn't thread safe. Many initializers look like PROCESS::<$EXECUTABLE-NAME> := $*EXECUTABLE.basename;, i.e. they modify a shared array 15:16
15:22 evalable6 left, linkable6 left 15:25 frost left
MasterDuke just need to stick a $lock.protect around it? 16:03
nine I'm afraid that will neither be easy nor enough. Not easy because bootstrapping and not enough because as stashes are shared, they can be read by other threads not holding a lock. So we gonna have to lock and clone/replace 16:07
Looks like my fix at least works. No idea about the performance impact though. It surely won't make compilation or module loading faster. But then I'd guess that we don't bind that many Stash entries (more like hundreds instead of tens of thousands). So the impact may be negligible. 16:52
With that taken care of rakudo -e 'await (^5).map({start { print qqx{echo $_} } })' seens remarkably stable. IF the JIT is disabled. I had hoped to get better debugabilty by disabling the JIT. Didn't think that the segfaults are actually caused by some JIT issue 16:53
MasterDuke what about just the expr jit disabled? 16:59
nine Checking...will have to give it an hour or so to gain any significance considering that at least with the rr runs I got just 3 segfaults in 30K runs 17:06
lizmat nine: re INITIALIZE-DYNAMIC... those are only supposed to run in the main line of the setting ? 17:08
well.. REGISTER-DYNAMIC is supposed to run only in the mainline of the setting, aka on a single thread ? 17:11
ah. do you mean the access to PROCESS:: is not thread safe ? 17:13
nine yes the access 17:28
lizmat ok, will look at fixing that 17:33
nine lizmat: already in the works 17:57
lizmat ah?
nine One spectest failure: Cannot clone an MVMContext
lizmat so how do you intend to fix it ?
nine As I said above: add a lock and clone/replace the $!storage in ASSIGN-KEY 17:58
lizmat so that would apply to *all* PseudoStashes? or just the PROCESS:: one ?
nine Stash 18:00
I figure Stash in general needs the fix as stashes are pretty much global data that's frequently read and also mutated 18:01
lizmat fair enough
18:02 reportable6 left 18:03 reportable6 joined
nine Ah, this error points out that we don't have to clone for all ASSIGN-KEYs. Only needed when it's a new key (i.e. we actually mutate the hash). 18:06
lizmat well, in the case of INITIALIZE-DYNAMIC, it most certainly is a new key
fwiw. I'm working on putting the lockig logic in INITIALIZE-DYNAMIC 18:07
*locking
nine I don't think that's enough as it doesn't cover any other PROCESS::<foo> := $foo; 18:09
lizmat well, I'd assume those would be either in the mainline of the setting, or already set 18:10
but you're right, all access to PROCESS:: should be guarded as well...
nine grep PROCESS:: t/
meh... grep -r PROCESS:: t/ of course 18:11
lizmat uses ack -)
MasterDuke mostly uses git grep 18:26
nine MasterDuke: still get segfaults with expr jit disabled 18:30
MasterDuke huh, i'm wondering if there's a bad template somewhere, since my commit that jits ctx(call|out)er(skipthunks) causes problems even when applied against master (i.e., not just on the branch i have with other commits before it) 18:36
but everything works fine with expr jit disabled (even though i didn't add templates for those ctx* ops)
18:39 squashable6 left
lizmat nine: scratched the refactoring INITIALIZE_DYNAMIC idea 19:07
it builds fine, but then dies installing modules in a *very* strange way :-( 19:08
can't deal with such madness atm -)
19:24 linkable6 joined 19:25 evalable6 joined
nine Indeed, looks like it's perfectly stable when the JIT is disabled 19:41
19:42 squashable6 joined
nine lizmat: just created a PR with my fix. I wonder if you could have a look if it slows down anything noticably? 19:56
lizmat nine: OOC, shouldn't AT-KEY not also be locked 19:58
?
nine commit message says no ;)
lizmat ah, duh 19:59
is cloning a hash not threadsafe? 20:00
nine no 20:01
lizmat ah, yes, if you don't do that in the protected block, you might use other thread's updates
nine or even half baked updates
also when multiple threads clone, update and replace without locking we may lose some of those updates 20:02
dogbert11 nine: FWIW, your PR does not fix the hyper bug. To be fair that's another problem entirely :) 20:10
nine dogbert11: what hyper bug?
dogbert11 i wrote about it earlier, but in short t/spec/S07-hyperrace/basics.t is a flapper failing on test #33 20:11
it can fail even if spesh is disabled 20:12
lizmat nine: am pretty tired atm... will look at it tomorrow 20:13
nine dogbert11: which test is #33?
And now I got even a deadlock. How many bugs can a 48 character Raku program unveil?! 20:20
20:22 sena_kun joined
MasterDuke github.com/Raku/roast/blob/master/...ics.t#L141 20:22
dogbert11 ok 32 - Correct result of .hyper.map(*+1) (try 4)
ok 33 - .hyper.map({.fmt(...)}) on a Buf slice works
20:23 sena_kun left
dogbert11 nine: sounds like you've fallen into yet another rabbit hole 20:23
20:24 Altai-man left
nine dogbert11: yes, they seem to follow me around... Your test works on random data, which could explain the flappyness. But it's also running sprintf which uses a grammar in parallel. Both things worth checking out 20:29
dogbert11 I have removed the call to .pick locally but it still flaps 20:32
nine The my $assert_used_args; in nqp/src/HLL/sprintf.nqp looks fishy. I don't know what it does but it gets shared between all concurrent callers and it gets mutated during the parse 20:35
Maybe that needs to be a dynamic variable instead? 20:36
dogbert11 you mean my $assert_used_args; -> my $*assert_used_args; ? 20:42
nine defined in sub sprintf like @*ARGS_HAVE 20:43
It's been a lexical since 2013. Don't know if thread safety was much on the radar back then 20:44
dogbert11 wasn't threading stuff added later? 20:46
is it enough to rebuild rakudo if I change nqp/src/HLL/sprintf.nqp? 20:57
MasterDuke should be 21:01
dogbert11 I'm running it now. looks promising ... 21:02
Geth MoarVM/new-disp-nativecall-libffi: 38 commits pushed by (Stefan Seifert)++, (Nicholas Clark)++
review: github.com/MoarVM/MoarVM/compare/a...26d5fdf684
Nicholas nine: I think you need to "take over" new-disp-nativecall-libffi now. 21:05
It's new-disp-nativecall rebased onto master, with libffi implementations of all the dyncall commits
I think that there's more that can be removed - I don't think that anything in nqp or rakudo uses the OPs nativecallbuild nativecallinvoke or nativecallrefresh any more 21:06
which in turn means that (at least) MVM_nativecall_invoke can be deleted
I'm not sure what else can then be removed from the JIT code, and what other C code then goes
nqp & rakudo new-disp-nativecall builds on the JVM 21:07
NQP tests pass on master and it for JVM
rakudo tests are a bit of a mess. I think that that's a known issue
passes rakduo tests and spectests with ASAN and all the paranoia 21:11
nine Nicholas++ # massive! 21:12
Nicholas I "just" transcribed all your code. 21:13
Which wasn't actually that trivial. But it seems to be done now.
you did the hard part of making a working version
has only been tested on x86_64 21:14
(so far. might try some others tomorrow) 21:15
japhb Nicholas++ 21:17
21:30 evalable6 left, linkable6 left
nine I've just witnessed a race where JITed code is in the middle of setting up a call to a dispatch_monomorphic run_dispatch function while another thread updates the inline cache entry from a bogus pointer to something useful. Sadly just about 2 CPU instructions too late for the JIT code 21:33
21:33 linkable6 joined
nine 0x20b658421b80 <unlinked_dispatch> 21:39
That's the bogus pointer
21:49 squashable6 left 21:52 squashable6 joined
jnthnwrthngtn nine: That doesn't look bogus, that is what all callsites start out as? 22:08
22:26 CaCode joined 23:27 Kaiepi left 23:32 Kaiepi joined, evalable6 joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!