Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:55 frost joined 01:04 reportable6 joined 01:12 rakuUser left 02:12 bloatable6 left, squashable6 left, sourceable6 left, coverable6 left, reportable6 left, quotable6 left, nativecallable6 left, greppable6 left, benchable6 left, linkable6 left, notable6 left, statisfiable6 left, committable6 left, tellable6 left, bisectable6 left, shareable6 left, releasable6 left, unicodable6 left, evalable6 left, committable6 joined 02:13 nativecallable6 joined, greppable6 joined, quotable6 joined, tellable6 joined, benchable6 joined 02:14 linkable6 joined, squashable6 joined, notable6 joined, evalable6 joined 03:14 bisectable6 joined, shareable6 joined, coverable6 joined 03:15 reportable6 joined, sourceable6 joined 04:03 discord-raku-bot left 04:04 discord-raku-bot joined 04:15 unicodable6 joined, bloatable6 joined 05:13 statisfiable6 joined 06:02 reportable6 left 06:03 reportable6 joined 06:13 releasable6 joined
MasterDuke new (but don't know how recent) segv in new-disp, gist.github.com/MasterDuke17/8d228...c51e00bbad when using MVM_JIT_PERF_MAP=1 07:40
nine wasn't aware of MVM_JIT_PERF_MAP's existence 07:41
MasterDuke it's nice, i have it in an alias i use for profiling with perf 07:43
at `fprintf(tc->instance->jit_perf_map, "%lx %lx %s\n", (unsigned long) code->func_ptr, code->size, symbol_name);`, `code` is 0x0 07:45
nine MVM_JIT_DEBUG=1 shows why 07:47
It's 4e84cd01fea9970087a2060db1dd03a869398351 07:48
MasterDuke quick debugging 07:49
nine The error message about "dynamic label" made me suspect my change 07:50
MasterDuke MVM_jit_compile_graph checks `code` in the `/* Logging for insight */` if, should the jit_perf_map if check it also? or should `code` really just never be NULL? 07:53
nine Frankly, I don't know. Since the error messages are only printed when the debug flag is set, I don't know if it's "normal" for them to occur, or if they really should be more explody 07:56
MasterDuke fwiw, building rakudo on master with MVM_JIT_DEBUG=1 doesn't print anything 08:03
but there is a 'JIT ERROR: Negative offset for dynamic label 219' in t/09-moar/00-misc.t 08:05
08:05 linkable6 left, evalable6 left
nine So, not often but also not never 08:07
MasterDuke didn't see any during a spectest run 08:11
08:28 patrickb joined
Geth ¦ MoarVM: MasterDuke17 assigned to bdw Issue Rakudo test fail with MVM_JIT_DEBUG=1 github.com/MoarVM/MoarVM/issues/1535 08:29
jnthnwrthngtn moarning o/ 09:00
MasterDuke ahoy 09:05
jnthnwrthngtn So, what for today... 09:14
nine t/spec/S32-io/utf16.t runs into an infinite loop when run with MVM_SPESH_BLOCKING=1 MVM_SPESH_NODELAY=1
jnthnwrthngtn MasterDuke: One way to fix that bogus free is to tweak the calls in MVM_callsite_initialize_common to pass 0 as the final ("steal") argument to MVM_callsite_intern, although most ideally we'd only do it when --full-cleanup was set, which depends on your work in master, so maybe best to wait until the next rebase. 09:15
nine: Ouch 09:16
I might finally sort out istype today, so its two invoke calls can go away 09:17
MasterDuke jnthnwrthngtn: cool, i'll try and take a look at that after the next rebase
jnthnwrthngtn I'm kinda debating what to do with istype 09:18
09:20 lizmat_ joined, lizmat_ left, lizmat_ joined 09:23 lizmat left, TempIRCLogger left 09:24 lizmat_ left 09:27 lizmat joined 09:28 lizmat joined 09:29 TempIRCLogger joined
jnthnwrthngtn Probably the best way to preserve current performance is to 1) set it up like a dispatch up in the IC, but 2) have the istype op first look at the cache, and never bother with the ICE if it works out 09:29
Nicholas t/spec/S32-io/utf16.t (still) hangs for me (busy somewhere down in JITted code) with MVM_* all set 09:30
aha, what nine wrote
jnthnwrthngtn And then register a dispatcher per language which handles the other cases
MasterDuke IC(E)?
jnthnwrthngtn Inline Cache (Entry) 09:32
oh yay, NQP doesn't actuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
wtf, keyboard 09:33
NQP doesn't depend on anything other than cache-based type checking in its build
Nicholas yes, surely a proper malfunctioning keyboard in Prague would be rrrrrrrrrrrrrrrrrrr? :-)
jnthnwrthngtn :D
Anyway, this means I can replace the istype op in-place without worrying about doing a bootstrapping dance 09:34
Maybe it's 'cus this keyboard is a UK layout one... :)
OK, istype changes can be task 1 today 09:35
Nicholas .tell brrt Pyston (finally) seems to have solved its "how do we monetise this?" problem: blog.pyston.org/2021/08/30/pyston-...-anaconda/ www.anaconda.com/blog/pyston-team-...s-anaconda 09:39
tellable6 Nicholas, I'll pass your message to brrt
jnthnwrthngtn Nicholas: Is that the "join a company that is doing well enough so we can spend a portion of our work time on it" approach? 09:50
09:54 Nicholas left 10:07 linkable6 joined 10:08 evalable6 joined 10:20 Nicholas joined
Nicholas I should probably be careful when speculating on a publicly logged channel. 10:20
I observe that they had months between the first "here is 2.0 - it's closed source until we figure out how to monetize it" blog post 10:21
and the "hey, here it's open source"
and now "hey, we joined this other firm"
where they do say "We talked to a couple of companies about a possible joint future for Pyston, and Anaconda stood out to us in terms of alignment." 10:22
and also that "hey, here it's open source" happened just about the time that Facebook revealed (and open sourced) their Pythong ahead-of-time speedup thingy, cynder
jnthnwrthngtn
.oO( sounds pants... )
 10:23
Nicholas and that was also rouhly when Guido joined MS and they announced a plan to make Python faster
which wuold seem to obsolete some of the "business case" for "hey, fund us to work on Pyston"
(which is sort of sad, because seemingly Pyston is here, now, drop-in replacement and faster" while all the rest are "special" or "our roadmap predicts" 10:24
so it wasn't quite what you wrote, but "join a company that already has a working business model providing python-related speedups, so that we can actually work on code, rather than struggling to figure out how to sell this stuff" 10:28
which also seems to keep re-enforcing the observation that no matter how cool/useful your open source infrastructure speedup is, and how much money it would save businesses (individually or in aggregate) it's nigh on impossible to get them to spend money on it, when they thing they are already getting "good enough" for "free" 10:29
jnthnwrthngtn *nod* 10:37
10:56 squashable6 left 10:58 squashable6 left 11:06 sena_kun joined
lizmat nine: ok, creating a ThreadPoolScheduler object, does *not* start the supervisor thread 11:23
dogbert11 what might be happening here: moar: src/spesh/deopt.c:32: uninline: Assertion `MVM_callstack_current_frame(tc) == f' failed.
could it be something which interests nine?
lizmat nine: and the way it is now, it is run when the mainline of the setting is run
which is when it should take in any setting of RAKUDO_MAX_THREADS? 11:24
nine: so, I'm not sure what that has to do with compilation of the setting ? 11:25
nine lizmat: when is the mainline of the setting run? 11:27
lizmat at process start ? 11:28
*after* the compilation of the setting ?
nine But why is there no "Created scheduler" message then and why does it ignore RAKUDO_MAX_THREADS?
While adding my $*SCHEDULER = BEGIN PROCESS::<$SCHEDULER> = ThreadPoolScheduler.new; to the start of our service.p6 script fixes it? 11:29
lizmat lemme check some things out :-)
11:49 lizmat left 11:50 lizmat joined, lizmat left
nine lizmat: assuming that it is initialized at process startup, is $*ENV actually set up at that point yet? 11:51
tellable6 nine, I'll pass your message to lizmat
11:51 lizmat joined 11:53 TempIRCLogger left 11:54 lizmat left, lizmat joined 11:55 TempIRCLogger joined
MasterDuke lizmat: totally off topic, but i was curious while reading the weekly. do you know all the languages in the tweets you reply to? or do you just run them through google translate and assume it gets at least general meaning correct? 11:59
lizmat French / German I can read 12:00
tellable6 2021-08-31T11:51:26Z #moarvm <nine> lizmat: assuming that it is initialized at process startup, is $*ENV actually set up at that point yet?
lizmat anything else, I click on the "translate this tweet" link
and try to make sense of it :-) 12:01
nine: looks like a premature optimization is the cause of the issue 12:02
nine: running tests now
MasterDuke ha, nice 12:03
12:03 reportable6 left, reportable6 joined
lizmat nine: github.com/rakudo/rakudo/commit/b14d404a19 12:21
nine Cool :) 12:28
12:29 patrickb left 12:39 patrickb joined 12:42 psydroid left, AlexDaniel left 12:45 AlexDaniel joined 12:54 psydroid joined
jnthnwrthngtn Yay, I have istype switched over to look at the cache, then if it fails to fall back to using a dispatcher 13:01
Which can in turn do the appropriate wiring
nine So, istype is done? 13:02
jnthnwrthngtn Not quite
I've got the semantics right, or at least, NQP tests are happy and Rakudo ones are no less happy.
Need to put the JIT back 13:03
MasterDuke i don't remember, will new-disp make coercion parameters faster? see my question in #raku-dev for context
jnthnwrthngtn And hopefully that will explain the slowdown in CORE.setting compilation, but if not I'll need to figure that out also
MasterDuke: I don't know how they're compiled at the moment, so hard to say if anything already done would do so 13:05
13:06 rakuUser joined
MasterDuke hm 13:06
jnthnwrthngtn MasterDuke: Otherwise they're in the category of "things new-disp can probably help us make faster" 13:07
(But we'll need to write the disaptch logic to make it happen)
MasterDuke if you're curious and since i see you're not in raku-dev colabti.org/irclogger/irclogger_lo...-08-31#l91 has my question and an example 13:08
jnthnwrthngtn m: say 1.68 / 1.03 13:11
camelia 1.631068
jnthnwrthngtn heh, I coulda eyeballed that :)
Yeah, I don't see why we can't elimiante the difference there
MasterDuke m: say 1.5 / 0.87 # what i see locally 13:13
camelia 1.724138
jnthnwrthngtn Nudge me to look at it after new-disp merge if it isn't already faster 13:15
MasterDuke will do 13:16
m: say 2.4 / 1.7 # times for those example on new-disp right now 13:19
camelia 1.411765
MasterDuke better ratio, worse absolute times 13:20
jnthnwrthngtn Worse absolute times likely due to lack of inlining 13:23
MasterDuke the coerce version (on master and new-disp) spends a lot of extra time in github.com/rakudo/rakudo/blob/mast...#L115-L216 13:25
13:37 frost left
vrurg MasterDuke: BTW, the method is only used if a parameter was initially a generic type (after instantiation), or if argument type is not parameter target type. Otherwise I hoped new-disp will allow to re-write it as a dispatcher. 13:47
MasterDuke vrurg: fwiw, i see that as near the top in all my attempts at profiling coercion (though maybe those aren't representative of actual use) 13:48
vrurg If it gets involved, then there is barely something could be done to make this version inlineable, I'm afraid. 13:51
Neither I can think of any ways to optimize it. This version already cuts off any possibly unneeded branch of code. A couple of micro-optimizations possible, but they barely would make big improvements on your benchmarks. 13:52
nine I think dispatchers can make a big difference there as e.g. for sub foo(Int(Str) $i) {...}; foo("1") the dispatcher can determine the type of the passed parameter once, guard on that, and delegate to the .Int method directly (which can then be inlined at the callsite) 14:10
vrurg Exactly what I hoped for. 14:11
jnthnwrthngtn Phew, the istype JITting recovers the perf, I don't have another thing to hunt down 14:13
Geth MoarVM/new-disp: 12bb950997 | (Jonathan Worthington)++ | 19 files
Re-work istype in terms of the dispatcher

All cases that were previously resolved via the type check cache are still resolved in this way. All other cases are delegated to a per HLL dispatcher to sort out. This most immediately eliminates reliance upon the legacy method cache and invocation protocol, but also will open up some further optimization opportunities.		 14:21
MoarVM/new-disp: 73f261bdf5 | (Jonathan Worthington)++ | 6 files
Remove last lookup code for legacy method cache
 14:35
nine jnthnwrthngtn: the "cache-mess"? :) 14:37
jnthnwrthngtn :) 14:38
Can stop publishing those legacy method caches on NQP and Rakudo now 14:39
MasterDuke i wonder what the final diff (num lines added vs num lines removed) will be for each repo 14:40
jnthnwrthngtn Also in size of compiled output. Removing the legacy method caches made CORE.c.setting.moarvm over a megabyte smaller 14:43
MasterDuke ! 14:44
jnthnwrthngtn If anybody has master build handy, I'm curious how big that file is on master
MasterDuke this? `2763259 Aug 31 09:53 CORE.c.setting` 14:45
jnthnwrthngtn No, blib/CORE.c.setting.moarvm
MasterDuke `30791472 Aug 31 10:36 ./blib/CORE.c.setting.moarvm` 14:46
jnthnwrthngtn whoa
Of course there may be a few more things in master
nine But only a very few
jnthnwrthngtn But on new-disp
`22891248 srp 31 16:40 blib/CORE.c.setting.moarvm`
nine Holy size reduction Batman!
MasterDuke nice
jnthnwrthngtn I guess the new dispatch ops really are quite a bit more space efficient :) 14:47
nine Despite the string args, which take 4 bytes each. OTOH that's just prepargs and the prefix for the very first arg 14:48
jnthnwrthngtn It's also because spesh plugins all needed the plugin invoke to decide on the plugin, and then another invoke for its result. 14:50
So there were two prepargs and arg lists for those
15:36 patrickb left
Nicholas not ok 38 - --rxtrace does not crash 15:41
# expected: ""
# matcher: 'infix:<~~>'
# got: "Cannot find method 'trace-on' on object of type NQPClassHOW\n
...
That was ./rakudo-m -Ilib t/05-messages/02-errors.t
oh, I'm out of date...
rebuild, retest, ... 15:42
jnthnwrthngtn I see that one too 15:43
Ah, I didn't know there was a test for that. :)
I wonder if anybody uses it
nine On that utf16.t infinite loop: turning off the JIT gives some more information. Because instead of looping it fails with "Attempt to return outside of any Routine" 15:50
Geth MoarVM/new-disp: 158680cbbc | (Jonathan Worthington)++ | src/core/frame.h
Add a function to the public API for now

So we can tweak a Rakudo extop to not depend on the legacy invoke protocol. (Various extops are moving towards being syscalls in MoarVM wrapped in a dispatcher to do the Raku-specific bits, but the thing in question is an epic hack that should ultimately go away.)
nine Oh, and it's of course about inlining 15:52
Which reminds me, jnthnwrthngtn: why are Inline End annotations on the instruction following the inline and not on the last instruction of the inlinee? 15:57
jnthnwrthngtn nine: I'm pretty sure it used to go on the last instruction of the inlinee and that changed... 16:06
Aha, so it changed in 22a249d6cf5 16:08
I guess the code-gen (probably JIT too) were already treating it as exclusive (e.g. emitting it prior to the instruction in question), so this was the easier change 16:09
nine The JIT emitted it after the instruction. That's why I had to fix it in 4e84cd01fea9970087a2060db1dd03a869398351 16:10
And the new code to find that following instruction needed fixing in d678b12069e468aebe7967e1bbeb59ef73cf12ab to not segfault on empty BBs. 16:12
So....in hindsight, I question that part about the change being easier :)
jnthnwrthngtn Apparently :) 16:13
No objections if you want to move it onto the final instruction 16:14
Oh...I wonder if there was also an annotation motion reasoning too
(When an instruction is deleted) 16:15
But I don't see why that can't work either way too
nine If there's ever a gap in the stream of bugs to fix I may have a look :)
Removing the return in the get-filename sub gets rid of the bug 16:27
This turns raku-rv-decont into the main suspect
It's actually trivially reproducible with just: MVM_SPESH_BLOCKING=1 MVM_SPESH_NODELAY=1 ./rakudo-m -e 'sub f() { return ".".IO.parent }; for ^100 { f }' 16:31
jnthnwrthngtn wonders if this could be in any way related to t/spec/S32-io/dir.rakudo.moar failing 16:33
nine t/spec/S32-io/dir.rakudo.moar also fails with spesh disabled 16:36
Just with a "No exception handler located for catch" instead
jnthnwrthngtn Done 16:42
oops, ww
16:57 squashable6 joined 17:22 rakuUser left 17:32 rakuUser joined
Geth MoarVM/new-disp: 95d1a5a4ec | (Jonathan Worthington)++ | src/disp/syscall.c
Add get-code-outer-ctx syscall

Which will allow us to eliminate a Rakudo extop.		 18:00
18:02 reportable6 left
jnthnwrthngtn We're now down to just 10 extops 18:03
18:36 sena_kun left 20:04 reportable6 joined
[Coke] new-disp failed t/05-messages/02-errors.t for me with a recent build (might not be up to date) 20:36
timo 10 ex, tops! 21:00
japhb 10 ex-tops? 21:05
moon-child 10 bottoms! 21:06
timo github.com/facebookincubator/cinder/ - could be interesting to look at what decisions they made 21:07
for example their jit will immediately jit every function called ever, but they let you supply a text file to steer that 21:08
clearly very tailored to the exact code bases they have
MasterDuke yeah. fwiw, i thought the issues in the cpython fork that gvr is working with had some good potential ideas we could steal 21:14
timo i seem to recall the fork but not that gvr works on that 21:19
MasterDuke yeah, paid by microsoft to make python faster 21:20
github.com/faster-cpython/ideas/is...dated-desc
and you might be just the right person to steal from them with your pypy experience 21:21
21:29 rakuUser left
timo hard to say, since pypy was a from-scratch re-implementation 21:31
21:43 MasterDuke left
[Coke] rebuilt, still getting the error on t/05-messages/02-errors.t 21:47
gist.github.com/coke/8a46d2cf77e03...adcfdd78cb
jnthnwrthngtn I'm pondering removing that (functionality and test), unless anybody finds it terribly useful (it prints out parse output as it goes, but if one wants to see the parse tree there's --target=parse, which is about the same info) 21:51
(Rather than spending the time fixing up something that I don't think is used by any end user of Rakudo, and that I don't know that anybody working on it is using either) 21:52
22:15 rakuUser joined 22:19 rakuUser left, rakuUser joined
[Coke] deleted code is debugged code. 22:33
22:49 MasterDuke joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!