[00:01] *** sourceable6 left [00:01] *** bisectable6 left [00:01] *** releasable6 left [00:01] *** notable6 left [00:01] *** coverable6 left [00:01] *** tellable6 left [00:01] *** evalable6 left [00:01] *** statisfiable6 left [00:01] *** greppable6 left [00:01] *** shareable6 left [00:01] *** bloatable6 left [00:01] *** benchable6 left [00:01] *** nativecallable6 left [00:01] *** unicodable6 left [00:01] *** linkable6 left [00:01] *** reportable6 left [00:01] *** committable6 left [00:01] *** quotable6 left [00:02] *** sourceable6 joined [00:02] *** linkable6 joined [00:02] *** committable6 joined [00:02] *** notable6 joined [00:02] *** benchable6 joined [00:03] *** statisfiable6 joined [00:04] *** bisectable6 joined [00:04] *** shareable6 joined [00:04] *** evalable6 joined [01:00] *** frost joined [01:01] *** bloatable6 joined [01:04] *** nativecallable6 joined [01:04] *** tellable6 joined [02:02] *** greppable6 joined [02:04] *** releasable6 joined [02:37] *** squashable6 joined [03:02] *** reportable6 joined [04:04] *** unicodable6 joined [05:04] *** coverable6 joined [06:02] *** reportable6 left [06:18] good *, #moarvm [06:19] jnthnwrthngtn: coffee is on the menu again? [07:03] *** quotable6 joined [07:04] *** reportable6 joined [07:20] why do the bots flap? [08:04] *** linkable6 left [08:04] *** evalable6 left [08:06] *** linkable6 joined [08:15] https://colabti.org/irclogger/irclogger_log/raku-dev?date=2021-08-17#l54 asked and (mostly) answered a little while ago in #raku-dev [08:21] *** TempIRCLogger left [08:21] *** TempIRCLogger joined [08:23] *** TempIRCLogger left [08:23] *** TempIRCLogger joined [08:23] "sometimes it gets triggered when nobody says anything :D" [08:23] oh, it's *our* fault :-) [08:24] *** tealecloud joined [09:06] *** evalable6 joined [09:06] good *, evalable6 :-) [09:31] ❤️ [09:36] *** Altai-man_ left [09:37] *** Altai-man joined [09:51] moarning o/ [09:52] Nicholas: I'm not sure whether to interpret "third day" as today or tomorrow, so will conservatively assume tomorrow. [09:56] \o [09:56] and instead feast on ice-cream and beer? [10:14] Something like that :) [10:14] I have mild cold symptoms for the last couple of days also, which probably isn't helping anything :/ [10:18] Let's see if I can get to the bottom of this infinite recursion in PDF... [10:19] I'm failing to find a good pun/joke about "but that would mean that it wasn't actually infinite" [10:30] https://www.recipegirl.com/how-to-make-iced-coffee/ [10:31] if jnthnwrthngtn gets desperate :) [10:56] Phew, got a tiny golf of the PDF callsame issue [11:03] Wow, looks like it'll be the first case of a mis-compile by the dispatch program compiler in some time... [11:05] Yup, recording is good, but produced dispatch program missed a literal string guard [11:18] yay [11:18] if this is sorted out, next are PDF::Content and when it works - PDF::Class. [11:22] ¦ MoarVM/new-disp: e850b08c31 | (Jonathan Worthington)++ | src/disp/program.c [11:22] ¦ MoarVM/new-disp: Fix warnings in debug output production [11:22] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/e850b08c31 [11:22] ¦ MoarVM/new-disp: 7b3cfea602 | (Jonathan Worthington)++ | src/disp/program.c [11:22] ¦ MoarVM/new-disp: Always emit guards on resume init state [11:22] ¦ MoarVM/new-disp: [11:22] ¦ MoarVM/new-disp: Even if there are literal flags in the capture, they were literal values [11:22] ¦ MoarVM/new-disp: at the callsite of the *initial* dispatch, but can most certainly not be [11:22] ¦ MoarVM/new-disp: assumed to be literal at the point we do a `callsame`. This led to the [11:22] ¦ MoarVM/new-disp: generation of dispatch programs without literal string guards, which [11:22] ¦ MoarVM/new-disp: would in turn lead to callsame in two different methods on the same type [11:22] ¦ MoarVM/new-disp: but with different names going to the wrong place. [11:22] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/7b3cfea602 [11:28] Yes, PDF passes in full with this fix [11:33] PDF::Content also passes in full [11:36] jnthnwrthngtn, what about PDF::Class? [11:36] Tests running right now, we'll see [11:37] if it works, then PDF::API6. [11:38] Yes, PDF::Class is good, so now onto PDF::API6 [11:39] Seems either the issue in PDF was the blocker for all of these, or the problems they had were already fixed [11:39] if PDF::API6 passes, then we're back to complex ones. [11:41] Yup, just passed [11:41] awesome [11:43] I guess another blin run tomorrow to make sure it agrees with all of these fixes would be good; I'm guessing we're perhaps down to single digit number of modules to look into now? [11:44] if we subtract ones relying on removed ops and potential flappers, I'd say yes [11:44] yes, can do a Blin run tomorrow [11:45] shoes can go back on! [11:45] I guess HTML::Canvas is one that remains [11:45] ah yes [11:45] and HTML::Canvas::To::PDF, but I suspect it's fine now [11:47] If anybody fancies retrying HTML::Canvas ('cus they already have the deps for it to hand), please do. It'd save me figuring out what apt package to install for one of the dependencies if it already works... [11:47] all these modules have the same author so there is a good chance that the fix works on the HTML modules as well [11:48] *** evalable6 left [11:48] *** linkable6 left [11:49] *** linkable6 joined [11:50] After lunch I'll see if I've enough brain to figure out spesh linking of resumable dispatch programs [11:50] Well, to do the last step of it, since I think I got much of it worked out already [12:02] *** reportable6 left [12:04] *** reportable6 joined [12:24] *** frost left [12:39] jnthnwrthngtn: does t/spec/S12-methods/defer-call.t still pass for you? [12:40] I see not ok 21 - Two methods of different names in a class both using callsame [12:41] I don't think that I did anything stupid, but that's what I *always* think :-) [12:42] despite accumulated evidence to the contrary [12:42] Altai-man: was this the error reported by Blin for HTML::Canvas? 'Type check failed in assignment to @*SEEN[0]; expected Mu but got Int (1)' [12:43] dogbert11, exactly [12:48] then I fear it's still present. Installing all deps was a nightmare. [12:50] *** evalable6 joined [12:59] Nicholas: Yes, it passes. You've pulled latest MoarVM? [13:18] Specifically, it covers the fix in 7b3cfea6 [13:19] Guess I can't put off the resume init state call stuff forever... [13:19] jnthnwrthngtn: correct, PEBKAC - had fetched, but had not checked out [13:32] Phew. [13:47] D'oh, I fail. My "save a register or two of space" idea in sp_resumption by not having dummy slots in the register list when we know there's a constant in the dispatch program...turns the O(1) read at resumption point into an O(n) per arg, and since we're liable to need all of them, thus an O(n) into an O(n^2). Granted on a usually small number, but the code will be far more ugly too... [13:50] ¦ MoarVM/new-disp: bc3af919de | (Jonathan Worthington)++ | src/disp/resume.c [13:50] ¦ MoarVM/new-disp: Eliminate a potential way to create SEGVs [13:50] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/bc3af919de [14:02] you're taking all nine's fun away :-) [14:24] Wouldn't want there to be a fun backlog after his holiday... :) [14:26] I'm sure this is co-incidence, but the local supermarket has offers on certain beer and certain ice-cream [14:36] :D [14:36] Little bit of a trip, alas [14:43] I think (from the promo advert) that one of the beers was Budvar, but the local branch wasn't stocking that. *Or* the particular ice-cream flavour that I wanted. [14:43] will have to travel further... [14:43] It's Billa. They're endemic round here. [14:43] so not *much* further to the next branch [14:44] "Billa. We were endemic before COVID!" [14:45] oh yes, [14:45] Corona was also on offer. But I prefer Budvar. [14:47] I was impressed that the Mexican restaurant I went to while on vacation had not only Corona, but 4 *other* Mexican beers. [14:47] oh wow. [14:47] The two I tried were good. [14:48] Altai-man: HTML-Canvas-To-PDF works for me under new-disp [14:59] dogbert11, alright, then just HTML::Canvas is another difficult case. [15:07] yes [15:27] ¦ MoarVM/new-disp: e3028f1079 | (Jonathan Worthington)++ | 5 files [15:27] ¦ MoarVM/new-disp: Prepare init resume args for translated DPs [15:27] ¦ MoarVM/new-disp: [15:27] ¦ MoarVM/new-disp: When we translate a dispatch program into spesh ops, we'll also be [15:27] ¦ MoarVM/new-disp: translating each resume init into an sp_resumption instruction, which [15:27] ¦ MoarVM/new-disp: keeps the arguments and temporaries needed for resuption alive. This [15:27] ¦ MoarVM/new-disp: means the way we access the resume init args is quite different from the [15:27] ¦ MoarVM/new-disp: case where we had a dispatch recorded or dispatch run record on the call [15:27] ¦ MoarVM/new-disp: <…commit message has 6 more lines…> [15:27] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/e3028f1079 [15:27] ¦ MoarVM/new-disp: 19a13ac5f0 | (Jonathan Worthington)++ | src/spesh/codegen.c [15:27] ¦ MoarVM/new-disp: Fix off-by-one in sp_resumption code-gen handling [15:27] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/19a13ac5f0 [15:28] With some local uncommitted stuff, I just have my first working example of it being able to reconstruct the resume init state from a translated dispatch [15:36] Altai-man: Which module failed with "nextsame not in the scope of a dispatcher" or some such? [15:36] I must just know why [15:37] Or at least, I've noticed something a bit off [15:37] It was PDF::Class, but if you say it's fixed then no such module. [15:41] Oh. OK :) [15:44] *** japhb left [16:00] ¦ MoarVM/new-disp: 758b3e8a6f | (Jonathan Worthington)++ | 2 files [16:00] ¦ MoarVM/new-disp: Fully handle resumes in translated DPs [16:00] ¦ MoarVM/new-disp: [16:00] ¦ MoarVM/new-disp: With this, we can now start to translate dispatch programs that set up [16:00] ¦ MoarVM/new-disp: resume initialization state. This in turn makes them elligible for spesh [16:00] ¦ MoarVM/new-disp: linking. We do not currently, however, allow any inlining, either of the [16:00] ¦ MoarVM/new-disp: callee whose dispatch had resume initialization arguments or of the [16:00] ¦ MoarVM/new-disp: `sp_resumption` op. [16:00] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/758b3e8a6f [16:00] ¦ MoarVM/new-disp: f49640d1ef | (Jonathan Worthington)++ | 2 files [16:00] ¦ MoarVM/new-disp: JIT compilation of the sp_resumption op [16:00] ¦ MoarVM/new-disp: [16:00] ¦ MoarVM/new-disp: Only in the lego JIT now; trying to add this in the template JIT blows [16:00] ¦ MoarVM/new-disp: up, seemingly because it's not entirely happy we just ignore all of the [16:00] ¦ MoarVM/new-disp: operands except the first destination register. [16:00] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/f49640d1ef [16:01] That gets an amount of performance back [16:02] heck yea [16:03] *** japhb joined [16:03] Getting all of the inlining back should be the big win, though [16:03] (This is step one of allowing for that.) [16:03] for sure [16:04] spesh linking would get rid of calls to arg_guard_run, right? [16:04] Yes [16:04] not all of them, of course [16:04] but that was definitely something i measured a lot of in the random test code i had yesterday [16:05] And inlining is also two tasks: 1) make it work when we inline at a runbytecode when the dispatch has sp_resumption stacked up before it, 2) make it work when we inline sp_resumption itself [16:16] what part of this has the "when we see a spot that causes resumption, we can also inline the resumption code"? [16:37] now i'm thinking: should we emit enter and exit ops for dispatchers that have generated guards into our frames? [16:42] it's not really accurate to say we "call" the dispatcher every time its guards are used and succeed [16:43] so, there'd want to be a different kind of entry, perhaps [16:44] timo: I'm not attempting to translate dispatch programs that involve resumptions yet. [16:44] Just those that set up arguments for a resumption [16:45] oh, ok [16:45] That'll be possible later, but things building on it are *already* faster than in master [16:45] :D [16:48] i'm not even sure i know how "dispatch programs that involve resumption" look like [16:49] before: Executed in 8.40 secs fish external [16:49] Typically start with ops like MVMDispOpcodeStartResumption [16:49] after : Executed in 5.77 secs fish external [16:50] 1.84% time spent in spesh_arg_guard_run [16:50] 8.9% self time in MVM_frame_dispatch [16:52] went from 18.35% time spent in disp_program_run to 0.25% (0.31% inclusive) [16:52] Very nice :) [16:52] So "just" inlining to get back [17:11] ok so here's a thing [17:12] when you press "annotate" on a symbol that comes from the perf-blah.map from /tmp, perf will run this command for you /usr/bin/objdump --start-address=0x00007fdae823c000 --stop-address=0x00007fdae823e215 -l -d --no-show-raw-insn -S -C /tmp/perf-2006912.map [17:14] the arguments are -l = line-numbers, -d = disassemble, -S = source, -C = demangle [17:14] so, how exactly does the perf .map file work for this purpose? [17:16] do we actually have like the symbol list starting at 0x0, then up to, like, null bytes i guess? and then at the actual addresses listed in the file the assembly appears? like as a sparse file? [17:16] https://elixir.bootlin.com/linux/v4.11/source/tools/perf/util/map.c guess i'll go read some source [17:20] jnthnwrthngtn: Boom! [17:21] +++ Compiling blib/Perl6/BOOTSTRAP/v6c.moarvm [17:21] '/home/nick/Sandpit/moar-SAN/bin/nqp-m' --module-path=blib --ll-exception --target=mbc --output=blib/Perl6/BOOTSTRAP/v6c.moarvm --vmlibs=dynext/libperl6_ops_moar.so=Rakudo_ops_init gen/moar/BOOTSTRAP/v6c.nqp [17:21] MoarVM oops in spesh thread: Malformed DU chain: writer sp_resumption of 89(0) in BB 1 is incorrect [17:21] Spesh of '' (cuid: 29, file: gen/moar/BOOTSTRAP/v6c.nqp:1133) [17:21] Callsite 0x7ff8d4fbb4a0 (0 args, 0 pos) [17:21] ... [lots] [17:39] *** tealecloud left [17:45] so what i've gathered from the code: the stuff we do with our perf-%d.map is already everything we can do with that format. another thing we could do is have it be a "DSO_BINARY_TYPE__JAVA_JIT", which we get by making the very beginning of the file an empty line, i *think* [17:47] no, that's wrong. i need getline on the file to return zero or positive, but set the line pointer to null [17:51] ok, we can't easily get that to happen, so all we can do is have the file contain no valid symbols / symbol-lines [17:54] no i read that wrong [17:54] we are actually outputting the java jit type successfully [18:02] *** reportable6 left [18:04] *** reportable6 joined [18:04] getting raw assembly into the same file would be a new feature that'd have to be put in perf i guess [18:07] *** tealecloud joined [18:12] *** tealecloud left [18:27] *** squashable6 left [18:29] *** squashable6 joined [18:50] <[Coke]> "callsite" added to the docs for the first time (under samewith) - do we prefer callsite, call-site, call site... [18:56] <[Coke]> going with the latter. [19:01] <[Coke]> (I know docs are kind of OT for moarvm, but I figured this was the right crowd for this particular one) [19:19] am i going to build a binding to libelf for raku? [19:35] https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/jvmti?h=perf/core been recommended to steal from this [19:36] ¦ MoarVM/new-disp: e246328c02 | (Jonathan Worthington)++ | src/spesh/disp.c [19:36] ¦ MoarVM/new-disp: Add missing writer for sp_resumption op [19:36] ¦ MoarVM/new-disp: review: https://github.com/MoarVM/MoarVM/commit/e246328c02 [19:50] d'oh, jitdump.h is GPL-2.0-only licensed, so shipping it with moarvm isn't an option [20:12] It's kindof weird to have a header file with a limited license ... is there a replacement? [20:18] https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/util/jitdump.h?h=perf/core it's a couple of structs :\ [20:21] jnthnwrthngtn: "All tests successful." and now it's carrying on to spectest [20:42] <[Coke]> build from yesterday still showing nativecall concurrent test issues on windows [20:43] <[Coke]> wonder if I can change the test harness to run *EVERY* test in the debugger. [20:56] gdb can debug process trees, but mostly by changing follow-exec-mode and follow-fork-mode correctly at the right time ... [21:00] do you think i can take these structs and enums and say it's fair use? [21:01] <[Coke]> were I at dayjob, I'd bounce that off the lawyers. [21:03] Yet Another Society doesn't own moarvm, do they? [21:04] https://moarvm.org/roadmap.html - i think we can make one or two changes here [21:05] i do believe our big integer stuff is being jitted better now? with the sp_ versions of add_I and friends? [21:06] add sub and mul definitely have a "if both are smallint, immediately calculate, if overflow, go to slow path" [21:09] and of course escape analysis is at least partially implemented with some more magic left as an exercise for the implementors [21:15] the MoarVM AST portion of the features page is also no longer right [21:15] since we don't have the MAST objects any more that we feed into moarvm's code, we spit out binary code directly from nqp code [21:16] should that just be kicked out? we do provide the nqp code that does the bytecode writing so that's something [21:20] say, in QASTOperationsMAST.nqp we have %core_op_generators with constant strings a couple of times [21:21] with a dispatcher we could make these cached and the generator codes could become inlinable [21:21] no clue how often the functions that use them are called in general, if this would help at all [21:23] we pre-fetch decont, goto, null, and set into "my &op_..." already. i wonder if a dispatcher would perform better than that? or really only for cases that show up only a few times, since we require a cache slot for each mention in the source code [21:24] pre-size-array could actually cache const_i64 and setelemspos locally rather than getting it twice [21:29] list_* really don't have to lookup the push_* op for every item in the list though, for real :D [21:36] "sub boxer" which we use to create a little closure for box_i, box_n, box_s, and box_u respectively, could take %core_op_generators{$blah} for the type_op and op parameters instead of looking it up inside the closure each time it's called [21:42] only some op generators need more from the $frame (the first argument after the last change to get rid of $*MAST_FRAME) than the .bytecode, i wonder if we can do anything here to take advantage of that so we don't have to call .bytecode over and over in some cases [21:43] surely .bytecode gets inlined everywhere and just compiles into the little sp_p6oget_o op, but still [21:58] i may have too low sampling frequency, but it looks like the hottest op generator of all is dispatch_o, followed by - at quite some distance in terms of other functions in between, which also includes C functions - const_s [22:11] *** squashable6 left [22:12] *** squashable6 joined [22:57] https://gist.github.com/timo/0f7a2430c26e431231eba1b551993dc8 [23:17] NBD just found how to make the #5 most highly sampled nqp method during core setting compilation to go from 0.3% to 0.01% [23:19] <[Coke]> Nice! [23:24] turns out there's a feature called "directives" that only the js backend seems to use, and when it's turned on, the linefileof method will look at @*comp_line_directives [23:25] when that is never set, i guess it's very expensive [23:29] who wants to "env MVM_JIT_PERF_MAP=1 perf record the line that compiles the core setting and see if linefileof is similarly high up the ranking there?