Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:01 sourceable6 left, bisectable6 left, releasable6 left, notable6 left, coverable6 left, tellable6 left, evalable6 left, statisfiable6 left, greppable6 left, shareable6 left, bloatable6 left, benchable6 left, nativecallable6 left, unicodable6 left, linkable6 left, reportable6 left, committable6 left, quotable6 left 00:02 sourceable6 joined, linkable6 joined, committable6 joined, notable6 joined, benchable6 joined 00:03 statisfiable6 joined 00:04 bisectable6 joined, shareable6 joined, evalable6 joined 01:00 frost joined 01:01 bloatable6 joined 01:04 nativecallable6 joined, tellable6 joined 02:02 greppable6 joined 02:04 releasable6 joined 02:37 squashable6 joined 03:02 reportable6 joined 04:04 unicodable6 joined 05:04 coverable6 joined 06:02 reportable6 left
Nicholas good *, #moarvm 06:18
jnthnwrthngtn: coffee is on the menu again? 06:19
07:03 quotable6 joined 07:04 reportable6 joined
Nicholas why do the bots flap? 07:20
08:04 linkable6 left, evalable6 left 08:06 linkable6 joined
MasterDuke colabti.org/irclogger/irclogger_lo...-08-17#l54 asked and (mostly) answered a little while ago in #raku-dev 08:15
08:21 TempIRCLogger left, TempIRCLogger joined 08:23 TempIRCLogger left, TempIRCLogger joined
Nicholas "sometimes it gets triggered when nobody says anything :D" 08:23
oh, it's *our* fault :-)
08:24 tealecloud joined 09:06 evalable6 joined
Nicholas good *, evalable6 :-) 09:06
AlexDaniel ❤️ 09:31
09:36 Altai-man_ left 09:37 Altai-man joined
jnthnwrthngtn moarning o/ 09:51
Nicholas: I'm not sure whether to interpret "third day" as today or tomorrow, so will conservatively assume tomorrow. 09:52
Nicholas \o 09:56
and instead feast on ice-cream and beer?
jnthnwrthngtn Something like that :) 10:14
I have mild cold symptoms for the last couple of days also, which probably isn't helping anything :/
Let's see if I can get to the bottom of this infinite recursion in PDF... 10:18
Nicholas I'm failing to find a good pun/joke about "but that would mean that it wasn't actually infinite" 10:19
dogbert11 www.recipegirl.com/how-to-make-iced-coffee/ 10:30
if jnthnwrthngtn gets desperate :) 10:31
jnthnwrthngtn Phew, got a tiny golf of the PDF callsame issue 10:56
Wow, looks like it'll be the first case of a mis-compile by the dispatch program compiler in some time... 11:03
Yup, recording is good, but produced dispatch program missed a literal string guard 11:05
Altai-man yay 11:18
if this is sorted out, next are PDF::Content and when it works - PDF::Class.
Geth MoarVM/new-disp: e850b08c31 | (Jonathan Worthington)++ | src/disp/program.c
Fix warnings in debug output production
 11:22
MoarVM/new-disp: 7b3cfea602 | (Jonathan Worthington)++ | src/disp/program.c
Always emit guards on resume init state

Even if there are literal flags in the capture, they were literal values at the callsite of the *initial* dispatch, but can most certainly not be assumed to be literal at the point we do a `callsame`. This led to the generation of dispatch programs without literal string guards, which would in turn lead to callsame in two different methods on the same type but with different names going to the wrong place.
jnthnwrthngtn Yes, PDF passes in full with this fix 11:28
PDF::Content also passes in full 11:33
Altai-man jnthnwrthngtn, what about PDF::Class? 11:36
jnthnwrthngtn Tests running right now, we'll see
Altai-man if it works, then PDF::API6. 11:37
jnthnwrthngtn Yes, PDF::Class is good, so now onto PDF::API6 11:38
Seems either the issue in PDF was the blocker for all of these, or the problems they had were already fixed 11:39
Altai-man if PDF::API6 passes, then we're back to complex ones.
jnthnwrthngtn Yup, just passed 11:41
Altai-man awesome
jnthnwrthngtn I guess another blin run tomorrow to make sure it agrees with all of these fixes would be good; I'm guessing we're perhaps down to single digit number of modules to look into now? 11:43
Altai-man if we subtract ones relying on removed ops and potential flappers, I'd say yes 11:44
yes, can do a Blin run tomorrow
Nicholas shoes can go back on! 11:45
jnthnwrthngtn I guess HTML::Canvas is one that remains
Altai-man ah yes
and HTML::Canvas::To::PDF, but I suspect it's fine now
jnthnwrthngtn If anybody fancies retrying HTML::Canvas ('cus they already have the deps for it to hand), please do. It'd save me figuring out what apt package to install for one of the dependencies if it already works... 11:47
dogbert11 all these modules have the same author so there is a good chance that the fix works on the HTML modules as well
11:48 evalable6 left, linkable6 left 11:49 linkable6 joined
jnthnwrthngtn After lunch I'll see if I've enough brain to figure out spesh linking of resumable dispatch programs 11:50
Well, to do the last step of it, since I think I got much of it worked out already
12:02 reportable6 left 12:04 reportable6 joined 12:24 frost left
Nicholas jnthnwrthngtn: does t/spec/S12-methods/defer-call.t still pass for you? 12:39
I see not ok 21 - Two methods of different names in a class both using callsame 12:40
I don't think that I did anything stupid, but that's what I *always* think :-) 12:41
despite accumulated evidence to the contrary 12:42
dogbert11 Altai-man: was this the error reported by Blin for HTML::Canvas? 'Type check failed in assignment to @*SEEN[0]; expected Mu but got Int (1)'
Altai-man dogbert11, exactly 12:43
dogbert11 then I fear it's still present. Installing all deps was a nightmare. 12:48
12:50 evalable6 joined
jnthnwrthngtn Nicholas: Yes, it passes. You've pulled latest MoarVM? 12:59
Specifically, it covers the fix in 7b3cfea6 13:18
Guess I can't put off the resume init state call stuff forever... 13:19
Nicholas jnthnwrthngtn: correct, PEBKAC - had fetched, but had not checked out
jnthnwrthngtn Phew. 13:32
D'oh, I fail. My "save a register or two of space" idea in sp_resumption by not having dummy slots in the register list when we know there's a constant in the dispatch program...turns the O(1) read at resumption point into an O(n) per arg, and since we're liable to need all of them, thus an O(n) into an O(n^2). Granted on a usually small number, but the code will be far more ugly too... 13:47
Geth MoarVM/new-disp: bc3af919de | (Jonathan Worthington)++ | src/disp/resume.c
Eliminate a potential way to create SEGVs
 13:50
Nicholas you're taking all nine's fun away :-) 14:02
jnthnwrthngtn Wouldn't want there to be a fun backlog after his holiday... :) 14:24
Nicholas I'm sure this is co-incidence, but the local supermarket has offers on certain beer and certain ice-cream 14:26
jnthnwrthngtn :D 14:36
Little bit of a trip, alas
Nicholas I think (from the promo advert) that one of the beers was Budvar, but the local branch wasn't stocking that. *Or* the particular ice-cream flavour that I wanted. 14:43
will have to travel further...
It's Billa. They're endemic round here.
so not *much* further to the next branch
jnthnwrthngtn "Billa. We were endemic before COVID!" 14:44
Nicholas oh yes, 14:45
Corona was also on offer. But I prefer Budvar.
jnthnwrthngtn I was impressed that the Mexican restaurant I went to while on vacation had not only Corona, but 4 *other* Mexican beers. 14:47
Nicholas oh wow.
jnthnwrthngtn The two I tried were good.
dogbert11 Altai-man: HTML-Canvas-To-PDF works for me under new-disp 14:48
Altai-man dogbert11, alright, then just HTML::Canvas is another difficult case. 14:59
dogbert11 yes 15:07
Geth MoarVM/new-disp: e3028f1079 | (Jonathan Worthington)++ | 5 files
Prepare init resume args for translated DPs

When we translate a dispatch program into spesh ops, we'll also be translating each resume init into an sp_resumption instruction, which keeps the arguments and temporaries needed for resuption alive. This means the way we access the resume init args is quite different from the case where we had a dispatch recorded or dispatch run record on the call ... (6 more lines)		 15:27
MoarVM/new-disp: 19a13ac5f0 | (Jonathan Worthington)++ | src/spesh/codegen.c
Fix off-by-one in sp_resumption code-gen handling
jnthnwrthngtn With some local uncommitted stuff, I just have my first working example of it being able to reconstruct the resume init state from a translated dispatch 15:28
Altai-man: Which module failed with "nextsame not in the scope of a dispatcher" or some such? 15:36
I must just know why
Or at least, I've noticed something a bit off 15:37
Altai-man It was PDF::Class, but if you say it's fixed then no such module.
jnthnwrthngtn Oh. OK :) 15:41
15:44 japhb left
Geth MoarVM/new-disp: 758b3e8a6f | (Jonathan Worthington)++ | 2 files
Fully handle resumes in translated DPs

With this, we can now start to translate dispatch programs that set up resume initialization state. This in turn makes them elligible for spesh linking. We do not currently, however, allow any inlining, either of the callee whose dispatch had resume initialization arguments or of the
  `sp_resumption` op.		 16:00
MoarVM/new-disp: f49640d1ef | (Jonathan Worthington)++ | 2 files
JIT compilation of the sp_resumption op

Only in the lego JIT now; trying to add this in the template JIT blows up, seemingly because it's not entirely happy we just ignore all of the operands except the first destination register.
jnthnwrthngtn That gets an amount of performance back 16:01
timo heck yea 16:02
16:03 japhb joined
jnthnwrthngtn Getting all of the inlining back should be the big win, though 16:03
(This is step one of allowing for that.)
timo for sure
spesh linking would get rid of calls to arg_guard_run, right? 16:04
jnthnwrthngtn Yes
timo not all of them, of course
but that was definitely something i measured a lot of in the random test code i had yesterday
jnthnwrthngtn And inlining is also two tasks: 1) make it work when we inline at a runbytecode when the dispatch has sp_resumption stacked up before it, 2) make it work when we inline sp_resumption itself 16:05
timo what part of this has the "when we see a spot that causes resumption, we can also inline the resumption code"? 16:16
now i'm thinking: should we emit enter and exit ops for dispatchers that have generated guards into our frames? 16:37
it's not really accurate to say we "call" the dispatcher every time its guards are used and succeed 16:42
so, there'd want to be a different kind of entry, perhaps 16:43
jnthnwrthngtn timo: I'm not attempting to translate dispatch programs that involve resumptions yet. 16:44
Just those that set up arguments for a resumption
timo oh, ok 16:45
jnthnwrthngtn That'll be possible later, but things building on it are *already* faster than in master
timo :D
i'm not even sure i know how "dispatch programs that involve resumption" look like 16:48
before: Executed in 8.40 secs fish external 16:49
jnthnwrthngtn Typically start with ops like MVMDispOpcodeStartResumption
timo after : Executed in 5.77 secs fish external
1.84% time spent in spesh_arg_guard_run 16:50
8.9% self time in MVM_frame_dispatch
went from 18.35% time spent in disp_program_run to 0.25% (0.31% inclusive) 16:52
jnthnwrthngtn Very nice :)
So "just" inlining to get back
timo ok so here's a thing 17:11
when you press "annotate" on a symbol that comes from the perf-blah.map from /tmp, perf will run this command for you /usr/bin/objdump --start-address=0x00007fdae823c000 --stop-address=0x00007fdae823e215 -l -d --no-show-raw-insn -S -C /tmp/perf-2006912.map 17:12
the arguments are -l = line-numbers, -d = disassemble, -S = source, -C = demangle 17:14
so, how exactly does the perf .map file work for this purpose?
do we actually have like the symbol list starting at 0x0, then up to, like, null bytes i guess? and then at the actual addresses listed in the file the assembly appears? like as a sparse file? 17:16
elixir.bootlin.com/linux/v4.11/sou...util/map.c guess i'll go read some source
Nicholas jnthnwrthngtn: Boom! 17:20
+++ Compiling blib/Perl6/BOOTSTRAP/v6c.moarvm 17:21
'/home/nick/Sandpit/moar-SAN/bin/nqp-m' --module-path=blib --ll-exception --target=mbc --output=blib/Perl6/BOOTSTRAP/v6c.moarvm --vmlibs=dynext/libperl6_ops_moar.so=Rakudo_ops_init gen/moar/BOOTSTRAP/v6c.nqp
MoarVM oops in spesh thread: Malformed DU chain: writer sp_resumption of 89(0) in BB 1 is incorrect
Spesh of '' (cuid: 29, file: gen/moar/BOOTSTRAP/v6c.nqp:1133)
Callsite 0x7ff8d4fbb4a0 (0 args, 0 pos)
... [lots]
17:39 tealecloud left
timo so what i've gathered from the code: the stuff we do with our perf-%d.map is already everything we can do with that format. another thing we could do is have it be a "DSO_BINARY_TYPE__JAVA_JIT", which we get by making the very beginning of the file an empty line, i *think* 17:45
no, that's wrong. i need getline on the file to return zero or positive, but set the line pointer to null 17:47
ok, we can't easily get that to happen, so all we can do is have the file contain no valid symbols / symbol-lines 17:51
no i read that wrong 17:54
we are actually outputting the java jit type successfully
18:02 reportable6 left 18:04 reportable6 joined
timo getting raw assembly into the same file would be a new feature that'd have to be put in perf i guess 18:04
18:07 tealecloud joined 18:12 tealecloud left 18:27 squashable6 left 18:29 squashable6 joined
[Coke] "callsite" added to the docs for the first time (under samewith) - do we prefer callsite, call-site, call site... 18:50
going with the latter. 18:56
(I know docs are kind of OT for moarvm, but I figured this was the right crowd for this particular one) 19:01
timo am i going to build a binding to libelf for raku? 19:19
git.kernel.org/pub/scm/linux/kerne...=perf/core been recommended to steal from this 19:35
Geth MoarVM/new-disp: e246328c02 | (Jonathan Worthington)++ | src/spesh/disp.c
Add missing writer for sp_resumption op
 19:36
timo d'oh, jitdump.h is GPL-2.0-only licensed, so shipping it with moarvm isn't an option 19:50
japhb It's kindof weird to have a header file with a limited license ... is there a replacement? 20:12
timo git.kernel.org/pub/scm/linux/kerne...=perf/core it's a couple of structs :\ 20:18
Nicholas jnthnwrthngtn: "All tests successful." and now it's carrying on to spectest 20:21
[Coke] build from yesterday still showing nativecall concurrent test issues on windows 20:42
wonder if I can change the test harness to run *EVERY* test in the debugger. 20:43
timo gdb can debug process trees, but mostly by changing follow-exec-mode and follow-fork-mode correctly at the right time ... 20:56
do you think i can take these structs and enums and say it's fair use? 21:00
[Coke] were I at dayjob, I'd bounce that off the lawyers. 21:01
timo Yet Another Society doesn't own moarvm, do they? 21:03
moarvm.org/roadmap.html - i think we can make one or two changes here 21:04
i do believe our big integer stuff is being jitted better now? with the sp_ versions of add_I and friends? 21:05
add sub and mul definitely have a "if both are smallint, immediately calculate, if overflow, go to slow path" 21:06
and of course escape analysis is at least partially implemented with some more magic left as an exercise for the implementors 21:09
the MoarVM AST portion of the features page is also no longer right 21:15
since we don't have the MAST objects any more that we feed into moarvm's code, we spit out binary code directly from nqp code
should that just be kicked out? we do provide the nqp code that does the bytecode writing so that's something 21:16
say, in QASTOperationsMAST.nqp we have %core_op_generators<blah> with constant strings a couple of times 21:20
with a dispatcher we could make these cached and the generator codes could become inlinable 21:21
no clue how often the functions that use them are called in general, if this would help at all
we pre-fetch decont, goto, null, and set into "my &op_..." already. i wonder if a dispatcher would perform better than that? or really only for cases that show up only a few times, since we require a cache slot for each mention in the source code 21:23
pre-size-array could actually cache const_i64 and setelemspos locally rather than getting it twice 21:24
list_* really don't have to lookup the push_* op for every item in the list though, for real :D 21:29
"sub boxer" which we use to create a little closure for box_i, box_n, box_s, and box_u respectively, could take %core_op_generators{$blah} for the type_op and op parameters instead of looking it up inside the closure each time it's called 21:36
only some op generators need more from the $frame (the first argument after the last change to get rid of $*MAST_FRAME) than the .bytecode, i wonder if we can do anything here to take advantage of that so we don't have to call .bytecode over and over in some cases 21:42
surely .bytecode gets inlined everywhere and just compiles into the little sp_p6oget_o op, but still 21:43
i may have too low sampling frequency, but it looks like the hottest op generator of all is dispatch_o, followed by - at quite some distance in terms of other functions in between, which also includes C functions - const_s 21:58
22:11 squashable6 left 22:12 squashable6 joined
timo gist.github.com/timo/0f7a2430c26e4...b551993dc8 22:57
NBD just found how to make the #5 most highly sampled nqp method during core setting compilation to go from 0.3% to 0.01% 23:17
[Coke] Nice! 23:19
timo turns out there's a feature called "directives" that only the js backend seems to use, and when it's turned on, the linefileof method will look at @*comp_line_directives 23:24
when that is never set, i guess it's very expensive 23:25
who wants to "env MVM_JIT_PERF_MAP=1 perf record the line that compiles the core setting and see if linefileof is similarly high up the ranking there? 23:29

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!