buggable ???????????? It's time for the monthly Accidental /win Lottery ???????????? We have 1 ballots submitted by 1 users! DRUM ROLL PLEASE!... 00:00
And the winning number is 23! Congratulations to jnthn! You win a can of WD40!
00:08 vendethiel joined 00:18 vendethiel- joined 01:52 ilbot3 joined 02:18 https_GK1wmSU joined 02:20 https_GK1wmSU left 02:29 vendethiel joined 02:41 MasterDuke joined 05:33 brrt joined 06:42 brrt joined
brrt good hi #moarvm 07:22
let me ask the stupid question today 07:24
how hard would it be, with lex/yacc in hand, and MoarVM as a library, to make a bootstrap-nqp 07:25
is that even remotely worth the effort
timotimo huh ...
brrt basically, i'm asking, how hard would it be to make a compiler for NQP in C, using MoarVM and traditional parsing tools 07:26
timotimo yeah 07:27
brrt that would allow us to rid ourselves from binary moarvm blobs in the nqp codebase 07:28
timotimo yeah 07:29
hmm. when you have a nqp parser you can potentially compile the qregex engine with that and sidestep having to write Yet Another QRegex Impl 07:30
nwc10 jnthn: callgrind for the setting for your branch paste.scsys.co.uk/564738 and for master paste.scsys.co.uk/564739 makes it look like the branch should be a winner 07:35
so maybe on the *big* stuff it is
07:35 zakharyas joined
timotimo today will be a day of insufferable heat :\ 07:38
moritz today will be a day of switching on the AC 07:40
yoleaux 31 Jul 2017 22:19Z <Zoffix> moritz: So "Mark Powers" from Apress emailed me a copy of the book; saying I should post a review on Amazon if I read the book and wanna review it... But where on Amazon do I post the review? I googled the book and found a link, but when I click review it it says "This item has not been released yet and is not eligible to be reviewed". I see a couple other reviews there already. What am I doing wrong? :/
timotimo if i had any
moritz: you know about the "protect somethingsomething relax" in one of the hyperlinks? the one for the | syntax 07:41
moritz .tell Zoffix I don't know how they managed that; a review when it's officially released would do nicely 07:42
yoleaux moritz: I'll pass your message to Zoffix.
moritz timotimo: I have no idea what you're talking about :(
timotimo 2 07:43
docs.perl6.org/type/IO\protect\cha...N\protect\
char"0024\relaxCOLONPath
one of at least two examples
nwc10 is afk for most of the day 07:51
brrt today is not that bad in .nl 08:14
08:20 domidumont joined 08:26 zakharyas joined 08:47 vendethiel joined
jnthn morning o/ 08:54
Urgh, hottest day of the year so far today, probably
nwc10: Hm, wow. Wonder why you get differnet results 08:56
Maybe my baseline was off or something
OK, so timing the spectests and the build it comes out about even 09:13
But
Master does 1281 GC runs 09:14
The branch does 1429
As a sanity check, if I never set the allocate_on_heap flag then it gives the same number of GC runs, so no oddities there 09:18
OK, I've managed to tweak the criteria so it's around 1310 or so 09:36
And that seems to be a wallclock winner
In part by bumping the threshold to decide but much more that I spotted we would bump the number of promotions whenever there wasn't a spesh cand 09:38
But that didn't imply we were spesh logging
So it could falsely inflate the number of promotions on frames where we didn't log the number of entries
Geth MoarVM/frame-heap-prediction: f29412f8ad | (Jonathan Worthington)++ | src/core/frame.c
Improve frame heap allocation prediction.

Only bump the counter of heap promotions when we have a spesh correlation ID, which means we will have bumped the entries; before this could get out of sync and over-state the frame's preference. Also increaes the thresholds some. This greatly brings down the number of added GC runs that this change caused by decreasing the number of false positives.		 09:44
09:45 zakharyas joined
jnthn I'm really curious about the callgrind numbers on this vs master, so will run those while I get on with other stuff :) 10:01
10:37 brrt joined
Geth MoarVM: 53000c87fb | (Jonathan Worthington)++ | 4 files
Attach type tuples to callsites.

We will be able to use this information to better optimize calls where arguments are passed in Scalar containers than is currently possible.		 11:09
brrt is watching the graal vm talk on the polyglot conference 11:10
it's interesting
but
i'm a bit in panic mode
jnthn 'cus you have a talk coming up too?
11:15 robertle joined
jnthn So, results of master vs my branch are in 11:18
Seems with latest improvements I see it as a win too; it's possible I got confused by updating Rakudo at some point and the setting grew in some way
Either way, 10 billion cycles less 11:19
m: say 218 / 228
camelia 0.956140?
jnthn On CORE.setting compilation
I think that's a win :)
Geth MoarVM: 21450df926 | (Jonathan Worthington)++ | 3 files
Allocate frames on heap that will need promotion.

By trying to use existing promotions as an indicator. This somehow seems to end up doing *worse* for the CORE.setting build, however it looks like it helps `make spectest` wallclock time a bit (though it's all within noise).		 11:20
MoarVM: 3d140fdb50 | (Jonathan Worthington)++ | src/core/frame.c
Improve frame heap allocation prediction.

Only bump the counter of heap promotions when we have a spesh correlation ID, which means we will have bumped the entries; before this could get out of sync and over-state the frame's preference. Also increaes the thresholds some. This greatly brings down the number of added GC runs that this change caused by decreasing the number of false positives.
jnthn So, rebased and pushed
lunch 11:22
brrt jnthn: yes 11:34
jnthn brrt: Good luck with it :) 12:09
brrt thanks :-) 12:10
thing is, i'm covering much of the same ground, i guess, only my approach is quite different
12:12 AlexDaniel joined
jnthn Righty, time to try and get spesh fully back on its feet again with regards to Perl 6 code 12:13
12:16 vendethiel joined
jnthn Grr, why do things always have to go wrong in 200+ BB frames :P 12:25
brrt bisect! 12:34
actually, you could make a spesh-bisect tool pretty simply, i'd think
provided you give them a sequence number and use blocking 12:35
jnthn Oh, it goes away wiht the JIT 12:37
Uh, with the JIT disabled
I wonder if padding a deopt offset of -1 is normal...
Ah, it's unused if we have no inlines 12:38
Taht's odd 12:42
Deopt one requested by JIT in frame 'variable_declarator' (cuid '322') (-1 -> 4384)
but
Deopt one requested by interpreter in frame 'variable_declarator' (cuid '322') 12:43
Will deopt 4398 -> 4378
I don't see why the target index should be different
ohh 12:45
So as background, I just enabled having decont as a deoptonepoint
So in the pre-opt graph there's 12:46
[Annotation: INS Deopt One (idx 119 -> pc 4378; line 2977)]
[Annotation: Logged (bytecode offset 4370]
getlex r10(74), lex(idx=1,outers=0,$ast)
[Annotation: INS Deopt One (idx 120 -> pc 4384; line 2977)]
decont r35(14), r13(74)
After optimization it's 12:47
[Annotation: Logged (bytecode offset 4370]
sp_getlex_o r10(74), lex(idx=1,outers=0,$ast)
[Annotation: INS Deopt One (idx 120 -> pc 4384; line 2977)]
[Annotation: INS Deopt One (idx 119 -> pc 4378; line 2977)]
sp_guardconc r10(74), sslot(7)
Note the deopt point moved
And now two are on the same instruction 12:48
brrt yes 13:02
that's … oh
that might upset some expectations
jnthn Yeah, seems that being sure to append rather than prepend the annotations will do it though 13:03
Geth MoarVM: a11b5f1e7e | (Jonathan Worthington)++ | src/spesh/deopt.c
Improve deopt logging output.
 13:08
MoarVM: 3932a0f749 | (Jonathan Worthington)++ | src/spesh/manipulate.c
Append, not prepend, moved deopt one annotations.

So that the earliest one takes precedence, not the latest one. Otherwise, we might skip over code that we really should run after doing the deopt.		 13:11
MoarVM: 53ff82e417 | (Jonathan Worthington)++ | 2 files
Turn decont into a deoptonepoint.

This means that we will guard types resulting from a decont, which should in turn help us do a better job of Perl 6 code without the dubious aliasing handling of before.
13:13 AlexDaniel joined
jnthn ahh...interesting problem 13:27
If we enter a hot loop - typically the main loop of a benchmark - at a point where we ran out of spesh log, then we'll miss an ENTRY record and similar for it 13:28
And so the OSR points will go nowhere
And we won't optimize it 13:29
brrt
.oO(Now here is nowhere)
jnthn ponders
The obvious heuristic this might be about to happen is when we enter a new compilation unit for the first time 13:30
13:43 ilmari[m] joined
nwc10 jnthn: not afk as much as planned. 13:45
window for your branch has MVM_SPESH_BLOCKING=1
other window that built master did not
jnthn ah 13:51
Geth MoarVM: a79f22b120 | (Jonathan Worthington)++ | 2 files
Force logging when entering a new compunit.

This ensures that we log the outermost loops of a compilation unit, and so will reliably be able to OSR hot tight outer loops of benchmarks.		 13:57
jnthn That works pretty effectively :)
vendethiel can't this function be called from multiple threads? 14:07
nvm
14:09 zakharyas joined 15:15 zakharyas joined, brrt joined 15:36 lizmat joined 15:49 AlexDaniel joined
Geth MoarVM: ee44adbe6a | (Jonathan Worthington)++ | 3 files
Also handle case of nearly full spesh log.

So that we can better collect data for new compilation units in this case also.		 16:01
MoarVM: 9b166512da | (Jonathan Worthington)++ | 7 files
Change the way we log returns.

The previous way was fragile and could lead to us getting bogus depth counts, which then confused the order that specialization was done in
  (not to mention anyone reading the spesh log). It also would block any
effort to try and carry over the simulated stack between processing of logs.
MoarVM: 10203d7196 | (Jonathan Worthington)++ | 3 files
Don't let compunit bonus logs inflate quota.
 16:23
MoarVM: 99332f234c | (Jonathan Worthington)++ | 2 files
Tweak compunit heuristic for EVAL/BEGIN-heavy code
jnthn Well, didn't quite end up doing exactly what I expected this afternoon, but still, improvements are improvemnets :) 16:26
Will work on invocation spesh tomorrow
16:38 zakharyas joined 16:48 robertle joined 17:20 zakharyas joined 17:25 hoelzro joined 19:16 markmont joined 19:41 zakharyas joined 20:20 AlexDaniel joined
markmont There was talk on perl6-compiler a few days ago about Rakudo Star on systems that enforce W^X, see near the end of www.nntp.perl.org/group/perl.perl6....16180.html 20:44
timotimo made a change to prevent MoarVM from breaking on systems that do not allow executable heaps: www.nntp.perl.org/group/perl.perl6....16195.html 20:45
timotimo yup, i'm here
markmont That still leaves executable stacks. One solution is to build MoarVM with libffi 3.1 or later.
timotimo yup 20:46
jnthn Pretty sure that comes via dyncall and is accidental; it's been worked on a bunch upstream.
markmont But I just looked into dyncall. As it turns out, updating to the latest (unreleased head) dyncall solves the execstack problem and all Perl 6 spectests pass, see gist.github.com/markmont/4a7578970...fafed7ecbc
timotimo whoa
that's cool
so we don't have to tell people to pass a non-standard configure flag
markmont Alternatively, we could just apply these changes to the version of dyncall MoarVM uses and, again, all spectests pass: hg.dyncall.org/pub/dyncall/dyncall/...f0b683918a 20:47
timotimo i do believe we can just bump up the dyncall submodule
markmont I'm just throwing this out there in case someone wants to either update the version of dyncall or just apply the patch.
jnthn timotimo: Yeah, I think so
timotimo i.e. no patches that dyncall doesn't have upstream
jnthn I think bumping our bundled version is fine. 20:48
markmont Here's the main stuff we get if we bump up dyncall:
no execstack needed, ARM64 support, PPC64 support, PPC32 support under Linux, and improved stability/reliability for Mach-O. 20:49
well, that's all in dyncall. But it could lay some foundations for subsequent MoarVM work.
jnthn Sounds worthwhile then :)
markmont One potential concern is there hasn't been a release of dyncall in ~18 months, we'd be using an unreleased head. 20:50
jnthn We're a bit before release, so now sounds like a decent time to bump to latest dyncall
True, though there's configure options for people who want to use an installed dyncall if packaging.
For those doing a source build, probably this just makes things work better. 20:51
markmont Let me know if I can do anything else at this point. So far, I've only tested on Linux x86_64. Otherwise, thanks for being open and looking into this! 20:54
jnthn timotimo: One question on your patch: I think it doesn't impact moar --version?
In that it'd still say "built with JIT support"?
timotimo oh
we can totally have a piece of output for that 20:55
jnthn Just that people will get a notable performance slowdown if they have their OS configured so we "can't" JIT
timotimo there's still that workaround with the temporary files
jnthn And it's nice if it registers in the --version
timotimo i'll patch something up for the --version
jnthn wonders if this is a common configuration
timotimo "built with JIT support, but operating system forbids it" or something
jnthn Temporary file sounds...uh...interesting
timotimo wellllllll, given i had four different programs crash on me in the background while i had that option enabled ... :) 20:56
jnthn hah :)
timotimo (thunderbird exploded, a google chrome extension or tab or something, something from kde, and something else)
jnthn I guess we'd have to be darn careful about perms on the file
timotimo we're supposed to create a temp file and immediately unlink it 20:57
(but also mmap it twice in different modes)
markmont Here's how dyncall does it using mmap(): hg.dyncall.org/pub/dyncall/dyncall/..._wx_mmap.c
jnthn Otherwise we risk turning a program with an path injection vuln into one that potentially has the ability to run arbitary machine code...
oh, /dev/zero :) 20:59
Cute :)
timotimo that seems clever
jnthn Yeah, that makes it sound less like a security hole waiting to happen :)
timotimo and systems that don't have /dev/zero probably don't have execmem rejection 21:00
jnthn I suspect this bites most people doing JITs at some point :)
timotimo i would assume so 21:01
i don't understand why this works but our code doesn't, though 21:02
what their code does is just have one mmap and that start out as RDWR and gets turned read + execute later
that's what we do? but mprotect explodes in our faces when we try that? 21:03
(it gently explodes, of course)
markmont: can you shed some light perhaps?
markmont timotimo: I'm out of my depth here I'm afraid, I'm more of a sysadmin than a programmer. But I'll look. 21:04
timotimo i'll point you at the code where we do the thing 21:05
github.com/MoarVM/MoarVM/blob/mast...pile.c#L68 21:06
though you probably saw the patch itself, too
and these platform functions live in ...
github.com/MoarVM/MoarVM/blob/mast...six/mmap.c
markmont timotimo: yep! And thanks for the links. I'll take a look at those and MVM_platform_alloc_pages() and MVM_platform_set_page_mode(). 21:07
oh, those functions are pretty simple :) 21:10
timotimo yup, really just prettier than ifdefs 21:14
MasterDuke jnthn: i just tried doing a --profile-compile of the rakudo build and got immediate segvs, but i thought you'd fixed that? 21:16
jnthn MasterDuke: I fixed some profiler stuff, didn't try anything that sizeable with it. I think timotimo fixed something recently also 21:19
MasterDuke fyi, i was on moarvm HEAD
timotimo yup i see it 21:20
it explodes in invoke_o 21:21
code is a null object there
21:24 geekosaur joined
markmont timotimo: I've looked at everything and for the case where the system has both mprotect and MAP_ANON, it looks like MoarVM is doing exactly what dyncall does. I've got to leave, but I'll poke at this in detail over the next few days if no one beats me to it. 21:25
timotimo thanks! 21:26
21:51 geekosaur joined 22:03 markmont joined 22:33 lizmat joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!