Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:04 reportable6 joined 00:23 ggoebel_ joined 00:38 linkable6 joined 00:39 evalable6 joined 00:50 ggoebel_ left
timo the post about new-disp doesn't even mention nativecall, which not 100% sure if we'll get much better runtime performance? 01:03
definitely the few bytes off of each frame because we'll use the new-disp / syscall invocation convention now 01:05
01:28 squashable6 left 03:09 coverable6 left, nativecallable6 left, statisfiable6 left, quotable6 left, evalable6 left, bloatable6 left, tellable6 left, linkable6 left, greppable6 left, bisectable6 left, releasable6 left, notable6 left, shareable6 left, reportable6 left, sourceable6 left, benchable6 left, committable6 left, unicodable6 left, linkable6 joined 03:10 statisfiable6 joined, shareable6 joined, coverable6 joined, benchable6 joined, tellable6 joined 03:11 committable6 joined, notable6 joined 03:12 evalable6 joined 03:29 squashable6 joined 04:10 releasable6 joined, unicodable6 joined 04:11 nativecallable6 joined 04:12 quotable6 joined, sourceable6 joined 05:10 greppable6 joined
Nicholas good *, #moarvm 05:10
05:11 reportable6 joined 06:03 reportable6 left 06:04 reportable6 joined 06:11 bisectable6 joined 06:12 bloatable6 joined
nine jnthnwrthngtn: See github.com/rakudo/rakudo/commit/1c...fba874a8cd for why %!quote_lang_cache became an attribute 06:59
jnthnwrthngtn: typo in dereference: "and so also would deference the Scalar and guard its" 07:24
MasterDuke: another answer to why the JIT doesn't root anything: simpliy because the code doesn't have any local variables that need rooting. Look at MVM_interp_run. Only 4 op implementations in total need MVMROOT and only one of those is JITed and that to a plain function call instead to the inlined implementation. 07:38
MasterDuke yeah, i was looking at one of the ones that does need it github.com/MoarVM/MoarVM/blob/mast...2992-L3009 07:39
Nicholas I'd never thought of it that way - effectively the JIT is "hand crafted" implementations of the code in the case statements in MVM_interp_run. No more. 07:40
MasterDuke what do you mean "no more"? 07:45
nine Nicholas: well that's how they get written :) It's really just looking at MVM_interp_run and translating that 07:46
Also how I debugged some of them. Just looking for translation errors. Nothing more indeed. 07:47
MasterDuke nine: see anything obviously wrong in gist.github.com/MasterDuke17/129ca...beeef36731 ? 07:49
i'm getting a segv in MVM_disp_program_run, src/disp/program.c:2872 07:50
where val.o is 0x0 07:51
oh, when i rebuild with --optimize=0, i get it in src/disp/program.c:2802, but it seems to be the same problem 07:52
08:02 ggoebel_ joined
Nicholas valgrind might help 08:16
MasterDuke hm, not giving me any new information 08:22
interesting, only two ops call MVM_malloc in their implementation, settypecache and setboolspec 08:46
nine MasterDuke: you simply never write anything into dst :) 09:21
MasterDuke ha! 09:23
wow
how many times did i switch between interp.c and emit.dasc and not see that? 09:24
nine That's why it's always good to have a second pair of eyes :) 09:33
MasterDuke and i even noticed it when writing the template, because i saw i'd need an explicit `(store \$0 (call (^func &MVM_repr_at_key_o) ...`
jnthnwrthngtn moarning o/ 09:49
nine: Ahh, I see. I wonder if we could somehow share it between EVALs in the same program, though. 09:50
(If it's not an EVAL at BEGIN time, of course)
Nicholas \o 09:51
nine jnthnwrthngtn: I'm pretty sure we could. The only open questions for me would be * how to share it (i.e. how to communicate the cache from one EVAL to the next when there is no outer World) and * how to avoid threading issues with the shared cache. 09:54
jnthnwrthngtn: the World's is_nested method may come in handy when determining whether it's safe to share or not. is_nested is true for BEGIN time EVAL 09:55
09:56 evalable6 left, linkable6 left 09:59 evalable6 joined
jnthnwrthngtn nine: Hmm, OK. Will ponder it, although in the immediate I'll work at making dispatch stuff cope better with the situation. 10:05
sena_kun good morning! 10:43
jnthnwrthngtn, psst, comma ping 10:44
MasterDuke it's mildly annoying that two segfaults are performed when running a spectest 10:46
dogbert17 jnthnwrthngtn: your two recent nqp changes brings the runtime of juerd's regressed test down from 38s -> 13s 10:52
jnthnwrthngtn Yeah, I saw similar. Also looks like a slight CORE.setting parse time reduction and also on installing core modules. 10:54
dogbert17 ha, I was just going to test that :) 10:55
jnthnwrthngtn I need a couple of small additions to dispatch programs to let me properly do the megamorphic-on-type handler 10:56
m: say 1.802 / 2.061 11:10
camelia 0.874333
jnthnwrthngtn m: say 25.757 / 34.961
camelia 0.736735
jnthnwrthngtn Interesting.
(Improvement on test-t vs test-t-20) 11:11
dogbert17 perhaps [Tux] system is heavily loaded atm 11:12
the test-t-20 numbers do indeed look quite good though 11:13
jnthnwrthngtn Well, the other interpretation is that the specializer and cache warmup doesn't really get done soon enough to have such an impact on test-t
That's about 13% improvement, lizmat saw around 20%, it's possible that there's a CPU difference coming into play here 11:14
(The specializer runs on a background threads; cache sizes and such can impact how little impact one thread has on another) 11:15
dogbert17 sounds more than reasonable 11:21
Geth MoarVM: MasterDuke17++ created pull request #1552:
Jit some ops that aren't terribly hot, but do cause bails in a spesh log		 11:29
11:33 evalable6 left 11:37 squashable6 left 11:38 squashable6 joined 11:57 linkable6 joined 12:02 reportable6 left 12:03 reportable6 joined
Geth MoarVM: 0d1dfb48f9 | (Jonathan Worthington)++ | 4 files
Implement dispatcher-track-how

Allows the creation of dispatch programs that read the meta-object of some other tracked value. No guards are put in place, since every object has a meta-object, and any dispatch program wanting to make the access to the HOW a constant can already easily do so by guarding on the type and then inserting the read HOW as a literal.		 12:15
12:33 evalable6 joined
MasterDuke ugh, windows failures in github.com/MoarVM/MoarVM/pull/1552 with no useful debugging information 12:33
Geth MoarVM: f07aed4b2e | (Jonathan Worthington)++ | 3 files
Allow non-constant lookup tables in dispatches

So we can read a table out of a meta-object and then look up a method in it. This will allow us to create dispatch programs for method lookups when we are megamorphic on invocant type.		 12:39
MasterDuke the other ci failure in github.com/MoarVM/MoarVM/pull/1552 was due to failing the check that `raku --full-cleanup -e ''` didn't leak anything, and i can repro that locally 12:41
it's from spesh codegen 12:42
github.com/MoarVM/MoarVM/blob/mast...gen.c#L210
nine: is that one of the things that's known to not be fully cleaned up yet that your asan fixes branch handles? 12:43
only happens under load (spectest running in the background)
timo say, don't our different caches cause types to stick around longer when we have long-running programs that create and later discard new types regularly? 12:52
MasterDuke huh, those resume_inits should be getting freed github.com/MoarVM/MoarVM/blob/mast...date.c#L72 12:54
nine MasterDuke: quite possible, don't have the time right now to check 13:00
MasterDuke think i found the problem 13:01
13:01 ggoebel_ left
nine even better :) 13:01
Geth MoarVM: MasterDuke17++ created pull request #1553:
Fix leak of resume init registers		 13:08
MoarVM: 729912418b | (Jonathan Worthington)++ | src/disp/program.c
Enforce type/concreteness guard on tracked table

It must always be a concrete thing with VMHash representation, otherwise we cannot safely look up in it.		 13:09
13:36 frost-lab joined
Geth MoarVM: 8efb75806a | (Jonathan Worthington)++ | src/disp/program.c
Debug dumping of new dispatch program elements
 13:38
MoarVM: 3c74a0d75c | (Jonathan Worthington)++ | src/disp/program.c
GC marking of HOW values in dispatch recordings
13:44 Altai-man joined 14:02 Altai-man left 14:14 Altai-man joined
Geth MoarVM: 5877af1c01 | (Daniel Green)++ | src/6model/reprs/MVMSpeshCandidate.c
Fix leak of resume init registers

When freeing a spesh candidates resumption inits, also free their init registers.		 14:18
MoarVM: 2c485cb56b | MasterDuke17++ (committed using GitHub Web editor) | src/6model/reprs/MVMSpeshCandidate.c
Merge pull request #1553 from MasterDuke17/fix_leak_of_resume_init_registers
14:26 frost-lab left
nine Rakudo build on PowerPC fails with: [ 202s] get/bindlex_ng NYI 14:27
[ 202s] at SETTING::src/core.c/traits.pm6:14 (<ephemeral file>:trait_mod:<is>)
timo got the endian swapping wrong perhaps? 14:30
hum, that got pretty far to be failing at endian swapping
nine Exactly the same on s390x, so it does feel like an endianness problem 14:36
build.opensuse.org/package/live_bu...tems/s390x
timo could be that endian swapping doesn't know about the new "have callsites inside the op and use it to have var args" thing, but again, that would have to explode much earlier, possibly even in nqp already 14:38
nine And nqp's build seems to get stuck on aarch64 build.opensuse.org/package/live_bu...RM/aarch64 14:43
and armv7l build.opensuse.org/package/live_bu...ARM/armv7l
dogbert17 yesterday I manage to build and run spectest on RPi4 14:46
*managed
ah, but endianness is the same as on x86_64 14:48
Nicholas I build this on a Pi 2 days ago: This is MoarVM version 2021.09-584-g962e835dc 14:58
at a guess the endianness bug likely will be writing .i32 or .u32 into a union and reading back .i64 or .u64 15:00
or transpose reading and writing
dogbert17 Nicholas: are messages like this suspect: 15:12
src/spesh/disp.c:68:20: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
return (MVMCallsite *)GET_UI64(args, 2);
Nicholas very
dogbert17 there are four in that file
Nicholas um, wait, only fairly
misread it as "cast from pointer to pointer" where they point to different sized integers 15:13
that's the endian killer
dogbert17 there are two variants among the four, the other is 15:16
src/spesh/disp.c:244:35: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
guard->operands[2].lit_ui64 = (MVMuint64)hll;
15:26 squashable6 left 15:27 squashable6 joined
MasterDuke how do i know when i should use `(a|d|q)word` before a `WORK[foo]`? 15:29
nine I think these particular examples are harmless, since the pointer is the thing that's smaller and the integer is used to store a pointer 15:32
15:42 ggoebel joined 16:08 ggoebel left
dogbert17 nine, thx 16:17
Nicholas so, under valgrind on ppc64, CORE.c.setting instead bombs out with 16:54
Attempt to read past end of string heap when locating string
without valgrind, yes, "get/bindlex_ng NYI 16:57
nine Nicholas: is it sensitive to spesh? 17:04
Nicholas ooh, good question
rebuilding currently with different error messages for getlex_ng and bindlex_ng 17:05
nine The most probable cause for the "get/bindlex_ng NYI" message is bytecode getting out of sync. I.e. us jumping to the wrong place in the bytecode and interpreting data as ops 17:06
Nicholas well, it's "getlex_ng NYI 17:10
lizmat confirms MQTT-Client test from 41 seconds -> 14 seconds 17:11
[Coke] gist.github.com/coke/4626a2760c314...794c55f2ac - 4 compiler warnings on windows, latest moarvm 17:12
(I see dogbert17 raised these earlier, oops) 17:13
Nicholas nine: I think we're past the point of the first crash with `export MVM_SPESH_DISABLE=1` 17:15
nine Does a deopt happen right before the crash? 17:16
Nicholas how do I figure that out?
actually, I'm not going to. 17:17
I'm going to see if I can get rakudo built
nine src/spesh/deopt.c:7
Nicholas and if so, enable spesh again
and try to run tests
17:17 Altai-man left, reportable6 left, greppable6 left, sourceable6 left, quotable6 left, nativecallable6 left, unicodable6 left, releasable6 left, discord-raku-bot left, discord-raku-bot joined
Nicholas splitters! 17:17
I built this on a Pi4: Welcome to Rakudo™ v2021.09-228-gdd2b274fd. 17:20
Stage parse : 567.291
(that's the ppc64 build with spesh disabled) 17:21
17:21 [Coke]_ joined 17:22 Altai-man_ joined, reportable6 joined, greppable6 joined, sourceable6 joined, quotable6 joined, nativecallable6 joined, unicodable6 joined, releasable6 joined 17:23 [Coke] left 17:25 squashable6 left
Nicholas tests! with spesh! 17:26
$ ./rakudo-m -Ilib t/02-rakudo/03-cmp-ok.t 17:27
Unhandled exception: if_o/unless_o were superseded by the general dispatch mechanism at SETTING::src/core.c/hash_slice.pm6:6 (/home/nick/Perl/rakudo/blib/CORE.c.setting.moarvm:postcircumfix:<{ }>)
...
only MVM_SPESH_DISABLE=1 fixes it. 17:34
nine if_o/unless_o have even lower op numbers than get/bindlex_ng 17:35
17:42 kjp_ joined, childlikempress joined 17:44 moon-child left, kjp left, Voldenet left, vrurg left
Nicholas this is a bit daunting 17:44
but at least it's now reduced to "somewhere in spesh"
17:45 vrurg joined 17:49 Voldenet joined
nine tools/jit-bisect.pl --spesh may yield some more information 17:50
japhb MasterDuke: Did you find the answer to your word size question? 17:51
MasterDuke no, i just added some that i thought might be needed and am seeing what CI has to say 17:52
japhb If not: dword == 32-bit integer, qword = 64-bit integer, aword = 32- or 64-bit pointer (*a*ddress word)
AH! 17:53
I see, you weren't asking which each was, you were asking *when they should be used at all*
Nicholas nine: thanks. It's running
japhb I'd say "doesn't hurt to be precise", though the consequences of getting the wrong size are worse than no size, most of the time. (At least, that was true in my old x86 asm days) 17:54
MasterDuke japhb: when was the primary question, but the details about what they mean help with figuring that out
if you feel like looking at asm, i've been working under the assumption that the windows fails in github.com/MoarVM/MoarVM/pull/1552 are due to something i added in emit.dasc 17:55
japhb Woah, couple few commits in that PR? ;-) 17:56
MasterDuke i can't stand how the windows jobs start last, when they take the longest to run
yeah, but now i wish i'd created the PR from the beginning, so maybe i'd have some info to bisect which commit is causing the fail 17:57
Nicholas SPESH Broken frame: 322. 17:59
SPESH Acquiring log: spesh-0322.txt
This type cannot unbox to a native string: P6opaque, Sub
at SETTING::src/core.c/allomorphs.pm6:274 (/home/nick/Perl/rakudo/blib/CORE.c.setting.moarvm:val)
...
what do I do now?
paste.scsys.co.uk/595922 - spesh-0322.txt 18:01
18:02 reportable6 left
MasterDuke arg, windows is still failing 18:03
anyway, afk for a bit &
18:03 reportable6 joined 18:27 squashable6 joined
Nicholas emit_guard() can either set guard->operands[2].lit_i16 and then guard->operands[3].lit_ui3 18:29
or just guard->operands[2].lit_ui32
no idea if that's suspicious - does the thing that reads from the union know to read the right size?
(will assume "yes" bceause if I change it I get a SEGV) 18:31
sena_kun jnthnwrthngtn, ping? 18:32
japhb MasterDuke: In your PR, why do some of the additions to src/jit/graph.c put register numbers into MVMint16 and others into MVMint32, and also some wrap the argument for an MVM_JIT_REG_VAL in braces, and some do not? 18:38
18:41 coleman-x joined, colemanx left
nine Nicholas: do you by any chance see a deopt in val right before the crash? 18:43
Nicholas I don't know how to answer that. When I stick a breakpoint on that line I get loads of hits 18:44
so I don't know how to answer "right before"
nine Have you enabled MVM_LOG_DEOPTS? 18:47
Nicholas no
nine src/spesh/deopt.c:7 18:48
Nicholas aha
sorry
CPU bound on the terminal trying to keep up with the output 18:50
aha, more useful in a file 18:55
Deopt one requested by interpreter in frame 'simple-args-proto' (cuid '196') Will deopt 178 -> 164 Completed deopt_one in 'simple-args-proto' (cuid '196')
Unhandled exception: if_o/unless_o were superseded by the general dispatch mechanism 18:56
(OK, that got mangled by irssi)
If it wasn't obvious, I am completely unfamiliar with this code. 19:01
codesections I'm running into a segfault in some concurrent raku code, but don't (so far) have a golfed example. Would I provide the project any benefit by opening an issue with the (very minimal) output from perl6-gdb-m, or should not open one until/unless I have some golfed code? 19:14
(by "very minimal", I mean it's basically just "__GI___pthread_mutex_destroy (mutex=0x0) at pthread_mutex_destroy.c:32 19:15
32 pthread_mutex_destroy.c: No such file or directory."
jnthnwrthngtn codesections: Does typing "where" produce more output? 19:41
Geth MoarVM: 6b70820317 | (Jonathan Worthington)++ | src/disp/boot.c
Only guard on HLL in lang-meth-call

Not on the type. Otherwise, we make it impossible to add megamorphic method dispatch handlers.		 19:43
MoarVM: d6c99f1665 | (Jonathan Worthington)++ | src/disp/program.c
Cope with lazy HOW deserialization
dogbert17 jnthnwrthngtn: btw, during the new-disp work, altai-man found a non new-disp related bug in concurrent::stack. Should I report it in the corresponding repo? 19:46
sena_kun jnthnwrthngtn, ping, can you please confirm one Comma-related decision? 19:48
codesections jnthnwrthngtn, it does (another 13 lines). Thanks, I didn't know to do so 19:51
jnthnwrthngtn sena_kun: um...for some reason Slack didn't send my messages to you :S 19:53
dogbert17: Yes, please
sena_kun jnthnwrthngtn, yes, it's the same for me, yet on my phone it works. can you PM me here?
codesections I think Slack has had some issues today status.slack.com/2021-09/06c1e17de93e7dc2 19:54
jnthnwrthngtn codesections: Those would make for a bug report we might be able to figure something out with
codesections Ok, will do
dogbert17 github.com/jnthn/p6-concurrent-stack/issues/4
20:04 camelia left, nine left 20:05 nine joined, timo left, camelia joined 20:08 timo joined
[Coke]_ lizmat et al.: cloudblogs.microsoft.com/opensourc...-projects/ 20:11
20:12 [Coke]_ is now known as [Coke]
nine Nicholas: oh, of course it's a little bit more complicated. The exception handling/reporting code itself also likes to deopt. So to really get the last deopt before the crash it's best to set a break point on the MVM_exception_throw_adhoc call that throws 20:27
Geth MoarVM: 7099cffb87 | (Jonathan Worthington)++ | src/disp/program.c
Allow lookup values to appear in resume init args
 20:28
20:33 ggoebel joined
jnthnwrthngtn I'm on megamorphic meth plus a patch or two, and the MQTT test is almost back to it's pre-new-disp execution time. 20:52
20:53 ggoebel left
Geth MoarVM: c940e0cbac | (Jonathan Worthington)++ | src/disp/program.c
Be sure to emit guards on HOW and lookup values
 20:53
jnthnwrthngtn afk for a bit 20:54
timo that was the test that was always disappointingly slow, right?
jnthnwrthngtn Yes
timo here's hoping we can get it faster through new-disp
jnthnwrthngtn The one that went from 7s to 50s or so
21:00 ggoebel joined 21:03 Altai-man_ left
MasterDuke japhb: good questions. copy pasto is the only answer i have 21:11
21:16 childlikempress is now known as moon-child
jnthnwrthngtn Pushed the megamorphic by type handling to NQP master. 22:01
MasterDuke the mqtt test seems a little faster, but now there are 483,564 deopts 22:03
not just seems, is faster 22:04
57.6k `Deopt one requested by interpreter in frame 'BUILDALL' (cuid '36')` 22:12
jnthnwrthngtn On my machine it's now at ~5.2s, whereas before new-disp it was at ~4.5s
When I first measured it after seeing the regresison report it was around 28s 22:13
timo is BUILDALL the stuff we generate?
jnthnwrthngtn Hm, or maybe worse, I can't remember
Depends. We generate it in Rakudo, but not in NQP
timo ah, it could come from that, i see 22:14
jnthnwrthngtn MasterDuke: Hm, I don't see those kind of deopt numbers on the latest profile I did of it 22:15
Let me do one more just in case
timo we are testing mqtt's t/regex.t?
jnthnwrthngtn Yes
I see a little under 5000 local deopts and a similar number of global deopts 22:16
MasterDuke oh wow. turns out i was on my branch that jits all those new ops
timo 10 seconds on my machine %)
MasterDuke only 6700 deopts on master
22:16 linkable6 left, evalable6 left
jnthnwrthngtn 10% of that benchmark is spent optimizing NFAs 22:17
MasterDuke huh. so jitting a bunch of ops makes the deopts increase by 100x. why would that be?
jnthnwrthngtn Wrong JIT of something leads to guard fail, maybe? 22:18
It's lucky that's all it leads to though
22:18 evalable6 joined 22:19 linkable6 joined
MasterDuke well, there is the segv almost right away in the nqp build on windows on that branch 22:19
jnthnwrthngtn Everything about the profile of the mqtt test says "build a better grammar engine that doesn't recalculate NFAs all over", though :)
MasterDuke next grant? 22:20
timo 67% interpreted frames %) 22:23
yeah, mergesubstates and optimize from QRegex are my top 2, and they are both basically not jitted at all 22:24
set_block_outer_ctx is the top 3, not jitted because of ctxouter 22:26
unfortunately, i see only successful compilations of mergesubstates 22:29
jnthnwrthngtn MasterDuke: I think "next grant" I apply for (which I need to very soon) will probably be some new-disp follow-ups and returning to working on PEA. 22:36
MasterDuke ah, that'll be nice too 22:37
jnthnwrthngtn Until new-disp was merged, I had 3 big ongoing things: new-disp, rakuast, and PEA. I...felt like I wasn't managing to get anything finished.
I'd rather knock PEA off that list before adding anything else to it :)
I'm resigned to rakuast being a long haul, if nothing else because the ecosystem consequences will likely be far more difficult to navigate than they were for new-disp. 22:38
timo you won't get dessert until you finish the PEAs you have on your plate
jnthnwrthngtn I like peas, so this isn't an issue for me. :)
I like peas, so this isn't an issue for me. :) 22:39
On the upside, I think rakuast will be easier for more folks to contribute to than new-disp has been. 22:40
MasterDuke how much more work is there for rakuast? 22:41
jnthnwrthngtn Well, it was passing 333 spectest files in full, and there's 1349 of them, so apparently quite a bit. 22:45
(it was passing that many last time I measured it, I mean)
moon-child I thought current narrative was '% spectests passing doesn't tell the whole story because of composition'?
jnthnwrthngtn moon-child: This is correct. The problem is that I don't know how to work out what the whole story is. 22:46
moon-child fair enough
MasterDuke ugh, i suspect the only way to find the bad commit in my PR is to try reverting them one-by-one. but i'll also disable all except the windows jobs in CI, so at least the testing should go quicker 22:53
but i think that'll be a task for tomorrow 22:54
jnthnwrthngtn Indeed, sleep time is coming 23:00
23:22 evalable6 left, linkable6 left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!