MasterDuke i reset to 68031489deb3ba456990f4f88233664d2e079a61, since the next one touched that section of frame.c 00:06
now instead of a segv i get `Cannot invoke this object (REPR: P6opaque; NQPMu)` 00:07
`at gen/moar/stage2/QRegex.nqp:2324 (/home/dan/Source/perl6/install/share/nqp/lib/QRegex.moarvm:parse)`
#1 0x00007ffff77d5748 in MVM_6model_invoke_default (tc=<optimized out>, invokee=<optimized out>, callsite=<optimized out>, args=<optimized out>) at src/6model/6model.c:451
#2 0x00007ffff7773ca5 in MVM_interp_run (tc=tc@entry=0x555555758950, initial_invoke=0x0, invoke_data=0x7ffff786b64d) at src/core/interp.c:950 00:08
which is actually the same line of invoke_o as causes the segv at HEAD 00:09
00:35 http_GK1wmSU joined 00:36 http_GK1wmSU left 00:58 yoleaux joined 01:52 ilbot3 joined
harrisi jnthn: that is interesting. I hadn't actually thought about that, but that could make things slightly easier. it really then just goes back to translating the semantics, which is awfully annoying. 02:53
jnthn: but also the most interesting part, I think. :)
03:28 camelia joined
Geth MoarVM: MasterDuke17++ created pull request #623:
JIT pow_I		 03:44
05:59 brrt joined 06:09 camelia joined
brrt ohai #moarvm 06:17
yoleaux 2 Aug 2017 19:45Z <nwc10> brrt: ASAN finds even-moar-jit very exciting: paste.scsys.co.uk/564761
2 Aug 2017 20:28Z <nwc10> brrt: first bad commit is 2dbb62f9cade2, "Add sp_decont op" - as the only non comment change in that is to add a template, I guess it reveals a latent bug
brrt nwc10, i guess so too
fortunately, i'm sure jit-bisect will tell me
Geth MoarVM/even-moar-jit: 30 commits pushed by (Timo Paulssen)++, (Samantha McVey)++, (Jonathan Worthington)++, (Bart Wiegmans)++
review: github.com/MoarVM/MoarVM/compare/1...b72a544f98		 06:26
brrt if you keep on updating NQP_VERSION in this rate, then i'm going to have to merge every other day 06:27
also, ASAN is decidely not as informative on os x, which is really frustrating 06:29
anyway, let's see what jit-bisect says 06:30
06:36 brrt joined
brrt openbsd-archive.7691.n7.nabble.com/...12186.html 06:41
or, "I'm wrong, but I'm famous, so who cares" 06:42
fuuuuuuuu
anyway. what if we try setting it to PROT_NONE before setting it to PROT_EXEC 06:46
luajit does the exact same thing we do... 06:48
nwc10 I did wonder if the kernel keeps any state on the region. hence whether one can side step all this "you can't get there from here" by simply going via a boring state 06:50
brrt well, apparently you can sidestep the issue by writing to a file, then mmapping that file as executable 06:51
so the kernel isn't that clever
07:13 brrt joined
brrt CORE.setting with ASAN under bisect is on the slowest bisect i'v seen 07:34
07:52 zakharyas joined
nwc10 brrt: what does jit-bisect do? 08:02
brrt are you coming to my talk at TPC, i'll explain all about it :-P 08:19
jit-bisect runs a command and bisects on frame number and subsequently on basic block number to pinpoint the broken piece of code 08:20
so the two envionment variables MVM_JIT_EXPR_LAST_FRAME and MVM_JIT_EXPR_LAST_BB act as 'brakes' on the expr JIT so that it doesn't do anything beyond those 08:21
with MVM_SPESH_BLOCKING I ensure that - in a single threaded application - the order of spesh and JIT compilation is deterministic 08:22
it occured to me that we could do the same thing with spesh
jnthn moarning o/ 09:02
brrt moarning
jnthn, a spesh-bisector, that seems like a decent idea, wouldn't it?
jnthn grr, hot weather = crappy sleep
brrt: Yeah, it'd save some hand work
brrt (the null-case for the bisect being to not optimize)
jnthn Though the one I hunted down last night woulda been tricky, the application just started spitting out wrong results at some point rather than crashing 09:04
09:04 zostay joined
jnthn Uff, some lag on this connection :S 09:07
brrt you can do it with any reproducible test case, just have a script that exits nonzero on the wrong ouptput 09:14
but yeah, crashes are definitively easier
jnthn Aww 09:15
A stress test of my current invoke branch blows up just after CORE.setting build
oh, in loading CORE.setting even
"Unknown string encoding 'utf8'" 09:16
huh
Sadly, wasn't the one I fixed last night 09:17
aww, also it seems it's a mis-compile of something 09:18
brrt ouch
metacircular systems do have their disadvantages...
jnthn Yeah... 09:19
Well, otoh it means we test our stuff against at least one significant size application :P
brrt that is very true 09:20
jnthn My hope is that I can provoke the crash in tsets
make test doesn't; time for make spectest
ah, good, seems like some spectests will 09:21
oh wow, SEGV 09:24
Curious, it's a crash inside of eliminate_dead in graph building 09:30
But it's when we're making a graph from an existing spesh candidate for the purpose of inlining
It's a bit surprising such a graph would have dead BBs
Huh, and yeah, we get a spesh fixup error 09:35
nwc10 brrt: yes, was planning to go to your talk at the TPC formarlly known as YAPC::Europe 09:44
jnthn ah, oops 09:49
Turns out that we do an inline
But we do it inside of a basic block that dies
(Due to other optimizations) 09:50
09:52 robertle joined
jnthn This isn't new, but we have gotten much better at inlining 09:52
Or much more capable of inlining
So I guess it stayed well hidden
Geth MoarVM/spesh-invoke: 5 commits pushed by (Jonathan Worthington)++ 09:55
jnthn That was just a rebase on master fwiw 09:56
Geth MoarVM/spesh-invoke: 079dc9e962 | (Jonathan Worthington)++ | 2 files
Avoid walking of end of BBs linked list.

Could happen when the last block was unreachable.
brrt nwc10: awesome :-) 10:06
dogbert17 jnthn: if you're bored I have a gist which might be of interest: gist.github.com/dogbert17/04de6ec9...04a0339c03 11:06
jnthn I'm quite un-bored (though a bit frustrated I'm bug-hunting in spesh rather than putting more opts in...) 11:08
dogbert17 more opts ++, was merely wondering if this is something 'new' 11:11
jnthn I don't think so
It looks like a data race
On SC lookup 11:12
dogbert17 I had to run the script many times before the problem showed up in ASAN
perhaps I should do some callgrinding instead 11:13
a week ago I made a gist containing callgrind results for a small program showing results for the then current camelia and the merged spesh-worker branch 11:28
I have now added a new run to the gist, this it's the current spesh-invoke branch. gist.github.com/dogbert17/6bb3eeb5...c7ab6f31a7 11:29
*this time it's
jnthn huh, dunno what's going on with that 11:33
It should be resolving the multi candidates again now
lunch &
11:45 markmont joined
markmont brrt: "what if we try setting it to PROT_NONE before setting it to PROT_EXEC" -- I don't think so. It looks like the Linux kernel won't allow PROT_EXEC unless the mapping is backed by a file, see github.com/torvalds/linux/blob/mas...ks.c#L3610 11:53
brrt: "you can sidestep the issue by writing to a file, then mmapping that file as executable" -- yes, but this isn't sidestepping, it is the recommended practice. 11:54
brrt i disagree, that's still very much sidestepping 11:55
if that's what selinux does, then selinux is doing it wrong
markmont brrt: The reasoning is that if you have the same file mapped twice, once as writable and once as executable, then code executing in the region won't be able to find the writable address.
brrt .. come again? 11:56
i don't parse
… okay
so
no
markmont brrt: I don't like this "solution" either; I'm just reporting what I found. Many people in the Mozilla bug agree with us. But, currently, SELinux with deny_execmem and NetBSD 8.0_BETA won't allow anything else.
brrt okay, suppose i have a mmap'ed file, and once i've mmapped it writeable, and once i've mmapped it execable 11:57
then i have a bit of writeable-executable memory, do i not?
markmont brrt: A bunch of projects, including Chrome V8, are just sticking with mmap()/mprotect() and saying that you have to have deny_execmem off or mark the NetBSD executable as wx-okay. 11:58
brrt and just saying 'well ASLR will make sure that an exploit will not be able to use that fact', then that's just nuts
what about luajit? i heard theo de raadt thought luajit was doing it right, and luajit is doing just what we do 11:59
markmont Like I said, that's what people in the Mozilla bug said, and I think they are right. But Firefox's nanojit went ahead with the temporary file mapping.
I haven't looked at luajit yet, but I have looked at OpenBSD 6.1 and it allows mmap()/mprotect(). 12:00
brrt i see
o.O
okay, at this point it seems legit just to push back to selinux
and/or netbsd
wrong things are just wrong
markmont That's what Chrome V8 is doing, as well as, I think, Java. 12:01
brrt okay
can't say i understand the full implications of that linux bit of code 12:02
anyway… i f you *do* want to implement a temporary-file-mmapping-solution, i'd be okay with that, but it'd be very nice if we could make it conditionally-compileable 12:03
so that people that aren't on 'secured' systems don't have their.. .security compromised
along with a configure flag that says —with-jit-security-theater 12:04
markmont Well, if the file is opened with O_TMPFILE, it never is linked into the filesystem tree and hence "shouldn't" be able to be compromised.
brrt hmmm
i'm not even going to ask why we have a distinction between MAP_ANON and O_TMPLFILE 12:05
:-)
markmont But that assumes that the underlying filesystem supports O_TMPFILE, which many do not. The Mozilla nanojit code (and the libfficode based upon it) try O_TMPFILE first but then fall back to a regular open() followed by unlink()
brrt okay, that makes me chuckle 12:06
markmont On the Linux side, I don't see this as a big deal. RedHat and Fedora currently ship with deny_execmem turned off by default and people who want to turn it on can either not run programs which do JIT or they can write a local policy to allow it just for the program(s) they want to whitelist. 12:09
jnthn back
markmont On the NetBSD side, mprotect() will fail unless the moar binary gets installed with a special flag that permits it. People on NetBSD systems should know this command and running it should be no big deal. 12:10
lizmat jnthn: will it continue to make sense to bind attributes to lexicals for hot loops (like a push-all) ?
jnthn lizmat: Yes 12:11
timotimo if you have an information disclosure vulnerability, which seems somewhat common as far as exploits go i suppose, then you can find the writable chunk that goes with an executable chunk very easily
lizmat jnthn: check
jnthn *sigh* We've really piled up the workarounds in spesh in the past, for lack of time to do things righter
What started out as an inline bug turned out to be in part 'cus we didn't really delete basic blocks from an inline that we then decided was unreachable anyway (workaround 1). Fixing that revealed that we weren't so aggressive in deleting basic blocks (workaround 2). Fixing that reveals that we don't delete blocks that have exception handler annotations in them (workaround 3) 12:14
lizmat the very definition of technical debt, I guess :-( 12:16
jnthn Yeah
It's good to clear these things up and we'll get better/smaller code as a result 12:17
Just means I'll end up spending a good chunk of the day on this instead of the invocation opts I expected 12:18
lizmat sometimes you have to take a step back before can take 3 steps forward :-) 12:19
jnthn True
And the problems are tricky
[Coke] jnthn++ 12:51
Geth MoarVM/spesh-invoke: 34b6e590be | (Jonathan Worthington)++ | src/spesh/codegen.c
Validate range of branch target in spesh code-gen.
 13:22
MoarVM/spesh-invoke: 4c919f0905 | (Jonathan Worthington)++ | 3 files
Anchor inlinee handlers to inliner entry.

So that they will be considered reachable and not eliminated once we start to apply more aggressive dead basic block removal.
jnthn Those are two in theory independent parts of a larger local set of patches, fwiw. 13:23
Phew, think I'm about there 13:35
13:41 AlexDaniel joined
Geth MoarVM/spesh-invoke: 309da4c8dd | (Jonathan Worthington)++ | 6 files
Fix and expand dead basic block removal.

The specialization graph builder does dead basic block removal before it computes the dominance tree, to avoid problems in the calculation. In some cases, however, it was finding and removing basic blocks in spesh graphs formed from previously optimized code. This was because the basic block removal was deliberately incomplete, to work around a ... (8 more lines)		 13:44
MoarVM/master: 9 commits pushed by (Jonathan Worthington)++ 13:51
jnthn There we go, merged.
The main thing we're missing now is the whole rw container checking/guarding that multi-dispatch also wants, which is why I think we miss out on some of them still 13:52
Also there's a few other fixes needed to the frame simulation done wiht logs
lizmat time to bump NQP and Rakudo ? 13:59
jnthn Mebbe, though I've got more improvements coming 14:00
Those were in a branch because for a while they made things worse rather than better. :)
lizmat but sooner the better wrt to testing, no 14:02
or would that create too much noise atm ?
Geth MoarVM: 67f92011c0 | (Jonathan Worthington)++ | src/core/args.c
Toss duplicate use of named arg checking.

There's no way to hit this from either NQP or Perl 6, it's totally harmless if it happens anyway, but having to support it gets in the way of a simpler, cheaper, handling of named arg flags in the event of a deopt, which will let us remove something that frustrates inlining.		 14:03
jnthn lizmat: Can bump a bit later on today :) 14:04
Well, can now if you want, just may as well get evening/overnight testing of all my changes :)
lizmat I'll wait 14:05
it's not like I don't have other things on my mind atm :-(
jnthn Impending TPC? 14:06
lizmat yeah :-( 14:07
14:51 brrt joined 15:06 brrt joined
brrt jnthn; is sp_decont actually invokish, as far as you know? 15:16
jnthn brrt: Yeah, it's just decont without logging 15:18
So they'd JIT exactly the same
Currently working on argument spesh fwiw 15:22
Trying to elimiante the need to track what args are used at all in a specialized frame
Which has taken me to specializing named slurpy args into the code to build the hash 15:23
brrt okay (a noninvokish variant would be cool, but probably too much to ask :-))
jnthn "It depends" :)
Ideally we can spesh noninvokish deconts we know enough about into something better than decont :) 15:24
brrt aye 15:26
ideally, we can specialize them in the JIT further, for those were it isn't practical to do in the interpreter :-)
timotimo i had started a tiny bit on devirtualizing decont for the guard tree 15:32
but it turns out we have to follow two pointers in a row for rakudo scalars 15:33
wasn't sure if it'd make sense making all tree nodes bigger to support that, and also wasn't sure how to store this in a sufficiently generic way into the container spec or repr or what
15:37 AlexDaniel joined
jnthn yay, got the slurpy named hash thing working 15:46
On the callee side it now just turns into ops to build the hash 15:47
16:29 robertle_ joined
timotimo cool, so a nqp::create and a few bindkeys with static strings for keys 16:29
Geth MoarVM: baaf8b6dfe | (Jonathan Worthington)++ | 16 files
More sophisticated named arg handling in spesh.

Previously, we would keep track of them using the bit field used by unspecialized named argument handling. Since in an inline the use of that could conflict, the op to do the tracking was `:noinline`. Thus an attempt was made to not include them if we had no slurpy args and no deopt points. However, recent spesh changes have introduced more ... (10 more lines)		 16:33
lizmat further lowering++ :-) 16:35
jnthn No regressions from that, or so my stressing of builds and spectest tell me 16:37
Geth MoarVM: 0df98ee0e7 | (Jonathan Worthington)++ | 8 files
Remove unused sp_namedarg_used op.
jnthn And yes, it does nail the inline that I mentioned in the commit message now :) 16:40
Alright, enough for now I think 16:45
timotimo fantastic 17:13
17:31 zakharyas joined 20:10 vendethiel joined 20:17 AlexDaniel joined 21:05 brrt joined 21:07 AlexDaniel joined 21:08 AlexDaniel joined 21:19 AlexDani` joined
Geth MoarVM: af52fca88b | MasterDuke17++ | src/jit/graph.c
JIT pow_I

It's slightly faster to call this from the JIT.		 22:35
MoarVM: f1bfca0317 | (Jonathan Worthington)++ (committed using GitHub Web editor) | src/jit/graph.c
Merge pull request #623 from MasterDuke17/jit_pow_I

JIT pow_I
MasterDuke jnthn: did you see my comments last night about trying to track down the --profile stuff? irclog.perlgeek.de/moarvm/2017-08-02#i_14960192 and a little after 22:37
i'm not sure that i found anything helpful though 22:38
jnthn Yeah, I fear the problem is happening "further back" though and that's just the upshot of it 22:39
MasterDuke that's kind of what i suspected, but lost the energy last night to keep going farther back 22:40
23:01 markmont joined
Geth MoarVM: c8eb5c9ce7 | (Jonathan Worthington)++ | src/io/asyncsocket.c
Make it possible to cancel an async socket reader.
 23:02

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!