github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
timotimo the profiler now links routines that have a SETTING:: path to github, including the line number 00:09
MasterDuke !! nice !! 00:14
timotimo MasterDuke: how do you feel about doing a test-ride before i make this Actually Public????
oh damn it, i thought i made utf8 work properly
timo2timo üöä
timotimo nope, only question marks
MasterDuke heh 00:18
and sure
timotimo are you on bitbucket?
MasterDuke i doubt it 00:22
timotimo www.sharedrop.io/rooms/56e33c31-51...5df0b8d0a4 00:23
wait, is that you? :P 00:24
MasterDuke i think so, you shared something for about 1s
but i didn't have time to click it
timotimo OK
i was worried maybe some foreign secret agent wanted to steal the data 00:25
MasterDuke happens all the time
a very reasonable concern
timotimo git clone -b master /home/me/tmp/file.bundle moarperf 00:26
<- did not yet know you could actually just clone from a gitbundle
you'll have to "npm run build" (which will set up a change watcher, bu you can just ctrl-c it later) 00:27
and then "cro run"
MasterDuke hm, do i install webpack with npm? 00:30
timotimo oh
MasterDuke or my os package manager? 00:31
timotimo does it install all you need when you just "npm install" in the folder?
MasterDuke seems like 00:32
timotimo cool, that should do it, then
oh 00:33
everything on the "welcome screen" is doing nothing at all
except the file input and "load file" button
i should probably remove those for the first public release, so people don't get horribly confused when nothing does anything
MasterDuke hm, where is the 'cro' binary/shell script/whatever? 00:38
timotimo snstall all the cro modules 00:42
except zeromq
also the moarvm heapanalyzer
MasterDuke man, installing all these dependencies is not a fast process 00:47
or course i'm also using 2018.06 for this, so i don't have all the recent performance increases
MasterDuke huh, none of those actually put a 'cro' in my path 01:02
timotimo it's not actually necessary to "cro run" it 01:03
timotimo you can also set MOARPERF_HOST=localhost MOARPERF_PORT=20000 perl6 service.p6 01:03
did you install the "cro" package with lowercase c?
MasterDuke well, i used what arch linux packages are available 01:05
timotimo oh
didn't know about those
MasterDuke Could not find Routes at line 5 in: 01:06
timotimo oh, -I lib
i hope you're not writing because you're enjoying it so much %) 01:15
timotimo now i'% a little worried 01:25
MasterDuke got distracted 01:30
timotimo it really needs a simple way to cut down the size of some of these tables %) 01:33
MasterDuke ugh, DBIish doesn't want to install because NativeHelper::Blob isn't installed 01:38
and there's no package for it
timotimo oh, huh?
anyway, i'm going to bed now. if it tells you that "routine with id ??? doesn't seem to exist", just switch to "routines" and click the "get routine overview" button %) 01:46
MasterDuke ok
timotimo maybe you're lucky and the sqlite dbiish part doesn't actually use NativeHelpers::Blob?
and you can just monkeypatch it out?
:\
MasterDuke timotimo: ⚠ moarperf Type check failed in binding to parameter '$filename'; expected Str but got Any (Any) 02:36
fwiw, that was just using `cro run` 02:37
timotimo MasterDuke: 08:04
-sub routes(HeapAnalyzerWeb $model, ProfilerWeb $profiler, Str $filename?) is export {
+sub routes(HeapAnalyzerWeb $model, ProfilerWeb $profiler, $filename?) is export {
jnthn I guess we're about at the point where we need to figure out guards and SSA versioning 10:22
So that the post-guarded thing has a new version
timotimo yeah, sounds good to me 10:23
jnthn The easiest way is to have all the guard instructions have a w(obj) where they "write" the guarded value (in fact they can do absolutely nothing since it'll be the same real register)
timotimo i only see two methods of making that work, and i like neither; give the guard ops one write and one read parameter, or handle them just like inc and dec 10:24
jnthn Right, like inc/dec is the other way
jnthn I don't like either of them too... 10:24
timotimo another is to have them act a little bit like PHI nodes, i.e. build fake info structs for all those ops 10:25
so they only have the extra argument during spesh
jnthn That feels...icky, but yeah, it's also an option
timotimo that way we'd at least not have to duplicate the ops in the oplist to have both one- and two-arg versions 10:25
jnthn I guess the other thing I worry a bit about is if some other opt goes and does a tree rewrite that changes the "input" arg to the guard, which would mean the input and output versions differ 10:27
uh, regsisters differ, even
timotimo ouch
that would be bad, yeah
jnthn Maybe we should just swallow the cost of making them all have a w(obj)
timotimo if we jit the result most of the time, it's not such a big hit 10:28
jnthn And in the interp honor the write
And in the JIT, spot that it's a no-op during code-gen so not do it
timotimo another option that comes to mind is split BBs after every guard and insert PHI nodes, but that also sounds unclean and icky
jnthn It's also a post-split operation
In that we insert guards having formed the CFG
timotimo hurm, so upping the versions is going to be "fun" 10:29
jnthn Yeah, there's no nice option here
But we really do want two versions
timotimo i agree
jnthn Otherwise we can't track when we have things that we already established vs. established from the guard 10:30
timotimo aye. the "from guard" flag isn't terribly much help there either
jnthn That's at the heart of why we can't eliminate guards we prove aren't needed, but also why spesh plugin guard insertion has to be so reluctant to add facts: because they might be taken as having an effect on uses prior to the guard. 10:31
timotimo yup 10:32
jnthn Which is why ASSIGN-POS ends up with the horrific sequence of guards 10:33
timotimo an optization pass based on walking the instructions backwards to see if something already established what a guard is guarding against would be another way; also not nice. 10:33
jnthn No, I'd rather handle it properly within the SSA form 10:34
timotimo yes
jnthn That's why I'm leaning towards the extra write arg, despite it's obvious inefficiencies
*its
timotimo at least it won't increase .moarvm size since it's only ever in post-spesh bytecode 10:35
jeez i could lie down for a nap already >_<
yeah, maybe i will 10:38
jnthn Rest well :)
timotimo thanks, good luck with the guard writing refactoring 10:39
surely it'll be hairier than i imagine already
jnthn Yeah, it's going to be tricky, though mostly at the "getting things straight during guard insertion" phase; everything after that should "just" fall out correctly 10:40
brrt \o 11:31
i found the leakage of inline spesh graphs
when we try to get a graph for a non-specialized frame, and we inline other frames during optimization, we leak the inlinee graphs 11:33
jnthn Aha! 12:12
good find
brrt++
Geth MoarVM/reguard: 5bcae477c0 | (Jonathan Worthington)++ | 10 files
Start splitting SSA versions at guards

We have so far used a single SSA version to represent both pre-guard data and post-guard data. Initially, guards were only ever inserted after the read of a value, so this wasn't a big problem. However:
  * Now we insert extra guards during spesh plugin optimization and
... (16 more lines)		 13:03
timotimo do we just use number-of-versions++ for the post-guard version now? 13:08
jnthn Actually for the pre-guard version in this particular case, 'cus it means we have less work to do :) 13:09
But yes, new version numbers are just "the next one" 13:10
timotimo ah
jnthn OK, now the hard part...I need to write a version splitting thingy 13:11
timotimo for some reason i thought we'd have to take the next one from the pre-guard one and bump the versions in the rest of the graph
jnthn Those two would be semantically equivalent
However, the first one is easier because at this guard insertion phase we didn't build the DU chain yet
Well, and even if we had, it'd still be more work to go through them all and update 13:12
Whereas here we just update the writer
timotimo right 13:13
jnthn Ah, seems a version splitter ain't too bad to write. 13:21
timotimo that's a relief 13:23
jnthn ssa++ 13:23
Though I realize the splitting I'm doing is a form of SSI
Kinda :)
timotimo hah, nice 13:24
i haven't actually finished reading that paper
timotimo tbh i stopped before it got to any juicy bits 13:24
dogbert2 timotimo: how was the nap? 13:25
timotimo i feel the same, only later
dogbert2 i.e. tired ?
timotimo yup 13:26
dogbert2 :(
timotimo but naps have a nontrivial chance of critical failure for me, which didn't happen
so that's nice
dogbert2 critical failure doesn't sound good 13:27
timotimo hah
i just took the term from pen&paper role playing
dogbert2 aha
ilmari critical failure of nap sounds like something you don't wake up from
dogbert2 indeed
timotimo it's not quite as serious in my case 13:28
diakopter dirt nap comes to mind 13:30
dogbert2 hmm, looks like one shouldn't run tests with MVM_SPESH_NODELAY=1 and MVM_SPESH_BLOCKING=1 atm
timotimo wow, that's a poetic way to say that
dogbert2 SEGV in cleanup_dead_bb_instructions (cleanup_facts=1, dead_bb=<optimized out>, g=0xb0844958, tc=<optimized out>) at src/spesh/dead_bb_elimination.c:32 13:32
32 ann->next = move_to_ins->annotations;
diakopter impressive 13:36
timotimo i don't quite understand why my nqp::while(1, nqp::stmts(...)) turns into an actual nested block (i.e. closure-taking) and Seq's from-loop and all that 13:38
jnthn Well, thre's a draft split_version 13:47
Geth MoarVM/reguard: b9ffe79ea9 | (Jonathan Worthington)++ | 4 files
Implement SSA split; restore speculative args opt

There were two ways to try and do this: rename everything looking back, or rename everything going forward. Going forward seems to be rather simpler to make work, especially since the tree going forward has not yet been optimized. So, that's done. Also if we do miss any renames, it's fairly harmless, because they would just miss the facts. (Note that the facts are not being added, for the reasons that triggered this work.)		 15:19
jnthn I did try going backwards first :)
timotimo hm, not even copying over the facts 1:1? 15:24
jnthn Will add that later 15:28
timotimo OK. it's probably not so important given it'll have the proper facts soon after i guess?
i'll try the lie down thing again perhaps :|
brrt jnthn++ - the pace of progress scares me a bit 15:41
jnthn All of spesh scares me a bit :P
brrt lol " MVM_VECTOR_DECL(MVMSpeshBB *, bbq);" 15:42
jnthn Ah, the name lives from when I was doing the pred approach and it *was* a queue 15:43
(Wanted a BFS)
First cut of re-instating spesh plugins seems to vaguely work 15:46
Will need a little further effort to make it do something useful
(That is, result in better code) 15:47
brrt :-) 15:48
jnthn Today the weather is a bit cooler 15:49
Well, quite a lot cooler
brrt yes, it is much nicer
much, much nicer
jnthn Yeah...figured I'd deal with this tricky task while it was nicer
It'll finally let me bring to a conclusion something I started on over a month ago thinking it'd not be so hard :P 15:50
Geth MoarVM/fork-safety: 5 commits pushed by (Bart Wiegmans)++ 15:54
MoarVM/reguard: a7a2ca6f7d | (Jonathan Worthington)++ | 2 files
Allow getting a new version of an in-use temp
 15:55
MoarVM/reguard: 6e7b215e24 | (Jonathan Worthington)++ | 2 files
Update spesh plugin guard insertion

With this, all of the guard insertion is now updated to use the new SSA versioned guards.
jnthn uh-oh, somebody touchned ConcBlockingQueue :) 15:56
brrt++
brrt :-) 16:01
jnthn Glad to say that the above is doing Rakudo/NQP builds and passing spectest with MVM_SPESH_BLOCKING=1 16:07
Geth MoarVM/reguard: 004d2f3573 | (Jonathan Worthington)++ | src/spesh/manipulate.c
Missing call to MVMVECTOR_INIT

Can't be sure of zeroed memory on the C stack.		 16:12
MoarVM/reguard: 431f33ef8f | (Jonathan Worthington)++ | src/spesh/plugin.c
Update the test register in plugins

So later guards and attribute access will happen with the earlier facts (although we aren't yet propagating facts fully enough for this to allow getattrs_o reduction in all cases).
jnthn And adding in the fact copying has some nice benchmark effects 16:20
And...causes a segfault in the NQP build :/ 16:21
jnthn But...not under gdb :/ 16:24
(gdb) p facts->log_guard 16:26
$1 = 4032350144
uummm...
jnthn Figured that out...spectesting by this point :) 16:41
Geth MoarVM/reguard: 1a22aaa4c9 | (Jonathan Worthington)++ | 3 files
Copy facts when splitting SSA versions

This means that we accumulate new facts on top of what was already known. With this, we get better optimization of attribute fetches in spesh plugin guards and avoid some duplicate guard insertion, which helps various some benchmarks, especially in array and hash stores.		 16:45
jnthn A hash store benchmark that at the start of the week ran 4.4x slower than Perl 5 is now down to 1.9x slower (so running in 45% of the time that it used to take) 16:48
diakopter wow 17:00
timotimo beautiful results 17:02
i'm sure jnthn already has a clue for the next 50% :) :) 17:04
jnthn Hmm, just got the lowest spectest time I've seen in a while too 17:05
timotimo \o/
jnthn And was clean
le merge
Geth MoarVM/master: 7 commits pushed by (Jonathan Worthington)++
timotimo (o/
\o)
jnthn I was fearing this was going to be a multi-day headache, so am glad to have got it done in the day :)
Dinner time; bbl 17:10
travis-ci MoarVM build errored. Jonathan Worthington 'Copy facts when splitting SSA versions 17:47
travis-ci.org/MoarVM/MoarVM/builds/414610362 github.com/MoarVM/MoarVM/compare/4...22aaa4c9c3
timotimo jnthn: how do you feel about this addition to the spesh log? gist.github.com/timo/8b990f5089834...7cf7d93478 18:04
i'm somehow having real difficulty printing out strings there, because i want to throw out any special characters (so the log format doesn't get disturbed) and limit it to a shorter length (so thhat ... yes) 18:35
jnthn What's added in particualr? :) 18:48
Oh, the names of BOOTCode instances perhaps
timotimo that's right, name and - almost more importantly - the cuuid 18:50
jnthn Yeah, I'm good with that change 18:51
So long as it copes fine with anon things with no nmae :)
timotimo i could also put name and cuuid into the stats output, whereever BOOTCode appears 18:52
oh, wait
that's only "it's a BOOTCode, and it's concrete"
not which value it is
but the "static values" section could get it 18:53
2 = Instance (BOOTStaticFrame) - '' (1448)
this is what it looks like when the name is unset i believe.
jnthn ok, that works :) 18:56
timotimo i'd also like to at some point introduce printing not only the cuuid but also the CU's name 19:00
since we tend to have multiple occurences of the same cuuid in the spesh log, and when they are both called '', it's annoying :) 19:01
timotimo the generated code for nom-ws's check for spaces is kind of dreadful :D so many p6bools in there 20:35
lizmat yeah, p6bools are evil 20:40
timotimo y'know, we could put True and False objects into the hll and have an hllbool op that lives in moarvm 20:43
then it'd be fine for spesh to know all about it
a random thing i'm now interested in doing: teaching the jit how to devirtualize nativeref decont_[ins] 20:44
at the moment a decont_i on a nativeref will first look up the ContainerSpec from the object's STable, checks if the type matches (i.e. int vs int), dispatches to the right fetch_i implementation - lexical, attribute, positional, or multidim, which then checks what kind of int (or whatever) it references (i.e. int8, int16, int32, or int64), and that finally pushes the result towards the register 20:49
lizmat wow
timotimo the jit currently just calls the exact same function that interp.c has, that's the one that starts by looking up the containerspec 20:51
hum. 20:54
gotta find a suitable benchmark first 20:55
timotimo wow, this isn't the right benchmark for this, or it's just way too cheap 21:15
timotimo i've been running a moarvm with --optimize=0 21:30
jnthn That'll not be so speedy :) 21:31
MasterDuke timotimo: which buttons did you say worked? (i got it running)
timotimo MasterDuke: once the profile is loaded, they all do, but if you refresh, you'll sometimes have to go to the routine overview and click "get routine overview" or something to make the call graph work 21:35
MasterDuke ok 21:36
i thought the "load file" button was a file picker at first, but now it's working if i enter a file name 21:37
how come some times are underlined? 21:39
and man, i do love those github links to the file/line 21:40
timotimo the underlined ones are microsecond rather than milisecond
i thought it'd be important for those to stand out so you don't confuse the two
given the symbol is kind of similar at smaller font sizes 21:41
(not all of us have retina displays
)
devirt is especially fast if you emit the devirted one and then the non-devirted one right after that
MasterDuke ok, makes sense
no way to sort?
timotimo not yet ;( 21:42
i think we're losing some facts upon inlining 21:48
there's a sequence that really should have the facts; getlexvia_o, guardconc, decont_i; the first two arguments to guardconc and the second argument to decont_i are versions 1 and 2 of that register and none of the versions have a single flag set on 'em 21:50
the frame before inlining has the facts, and the devirt of decont_i also succeeds there, but after it gets inlined, no such luck 21:53
do we rely on logging to give us facts about getlexvia_o and we're not looking for the facts in the inlined frame, or something?
or should we "just" analyze sp_guard* ops in facts.c? 21:54
jnthn: what do you think? 21:56
jnthn Yes, and note that the work that I did today makes that analysis a lot easier to do safely :) 21:58
Which is why it wasn't there yet :)
timotimo i interpret that as "go ahead and put all the guard ops into facts.c" :)
jnthn Or at least, why I didn't do it sooner
Yes, you can do so :)
timotimo yey
jnthn It's super safe to do it now :) 21:59
github.com/MoarVM/MoarVM/issues/939 is a total brain screw...what on earth is going on with it... 22:00
timotimo maybe i can find out something using rr
but maybe not before bedtime
facts.c would probably be a nice relaxing time and i think i'm slowly getting a headache
that issue is probably not very relaxing :) 22:01
jnthn I have one guess
Which is that we have struct padding
And the compiler issues a write for more memory than it really needs
hah, wat, re-ordering the struct makes valgrind spew more :) 22:02
timotimo fun!!
i like using pahole to see the exact structure of a struct that was compiled by the compiler 22:03
jnthn bah, memset of the region clears up the additional errors but not the original one :/ 22:04
timotimo oh, hm, we might not want to be using lit_i16 for spesh slots, but lit_u16 at least 22:11
jnthn Or even an sslot member of the union 22:13
timotimo right 22:14
i'd prefer that, so i don't have to remember whether it was 16 or 32 :)
Geth MoarVM: 68b31976dd | (Jonathan Worthington)++ | src/core/ext.c
Fix uninitialized memory in extops info

Fixes #939.		 22:17
timotimo so it was basically harmless? 22:18
just accidentally more frames being considered deoptifiable than should have been?
timotimo looks like i've done something wrong somewhere 22:32
when i have sp_guard_facts only invoked for sp_guardconc, and do copy_facts as well as set MVM_SPESH_FACT_CONCRETE in the flags, my short program ends with a huge stacktrace about an error in an END phaser 22:33
Some exceptions were thrown in END blocks: 22:34
No exception handler located for catch
Geth MoarVM: fba5c6b03a | (Jonathan Worthington)++ | src/spesh/dead_bb_elimination.c
Harden BB deletion annotation motion

Fixes #938.
timotimo even The Empty Program does that
jnthn Yeah, that sounds pretty wrong.
Two dogbugs down 22:35
timotimo taking out the copy_facts makes it no longer do the crash
i wonder, when would it be unsafe to just take the facts from the guard's source register and copy them over to the target register? the target register shouldn't have any facts on it, right? 22:36
jnthn Indeed, that sounds...odd
Maybe some opt is taking place that ain't safe
As a result of the added facts
Though I dunno which
timotimo oh, now that's interesting 22:37
travis-ci MoarVM build errored. Jonathan Worthington 'Fix uninitialized memory in extops info 22:37
travis-ci.org/MoarVM/MoarVM/builds/414719173 github.com/MoarVM/MoarVM/compare/1...b31976dd09
timotimo m: say 0x10d.base(2) 22:39
camelia 100001101
timotimo that'd be known type, deconted, concrete, from_log_guard
it's overwriting those flags - which are on the *target* register of the guard instruction, with either 0 or 4 in what i have, which is just deconted 22:40
MasterDuke timotimo: i just had a thought. i like the link to github for 'SETTING::' routines so much, what about the option to upload the script that the profile is for, so its routines can be linked to/shown? 22:43
timotimo yes. 22:45
it'd ideally have an html editor with perl6 syntax highlighting features
at least now all the decont_i are being devirtualized 22:46
now i can measure one step of devirt, and then two steps of devirt 22:47
oh, i should really use spesh_blocking 22:50
travis-ci MoarVM build errored. Jonathan Worthington 'Harden BB deletion annotation motion 22:55
travis-ci.org/MoarVM/MoarVM/builds/414723962 github.com/MoarVM/MoarVM/compare/6...a5c6b03a24
timotimo m: say (5520020241 + 5520145571 + 5520033088) / (5660022473 + 5659967605 + 5659991722) 23:02
camelia 0.9752777768
timotimo that's one step of devirt
Geth MoarVM: 554fe425da | (Jonathan Worthington)++ | src/spesh/frame_walker.c
Don't add base index when looking for lexical type

This information is looked up against the static frame, while it's only the dynamic frame that is subject to the base_index due to the potential for inlining. Fixes a read past the end of the static frame lexical registry buffer.
Also move `lexreg` back outside of the loop; it's not managed by the GC and so can't move.		 23:08
jnthn That'll do for today :) 23:11
timotimo will have results soon for the two-level devirt 23:17
m: say (5200172438 + 5200307378 + 5200343000) / (5660022473 + 5659967605 + 5659991722) 23:20
camelia 0.91877735794
timotimo that does sound like a Win Worth Having
m: sub foo(int $a is rw) { for ^10_000_000 { $a = $a + 1 } }; my int $a = 1; foo($a) 23:21
camelia ( no output )
timotimo this is the "benchmark"
m: say 4.67 / 5.17 23:25
camelia 0.903288
timotimo the time spent mirrors the Ir count pretty closely
jnthn Pretty nice improvement 23:28
travis-ci MoarVM build errored. Jonathan Worthington 'Don't add base index when looking for lexical type 23:35
travis-ci.org/MoarVM/MoarVM/builds/414731367 github.com/MoarVM/MoarVM/compare/f...4fe425da5a

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!