00:24
harrisi joined
01:50
vendethiel joined
01:52
ilbot3 joined
02:00
https_GK1wmSU joined,
https_GK1wmSU left
|
|||
harrisi | jnthn: You're Jonathan Worthington, I'm assuming? If so, is anything relating to your dissertation public? It sounds as though it is researching exactly a project I've been interested in working on. | 03:09 | |
jnthn: .. this is disappointing, but I found it actually. :) | 03:13 | ||
05:53
brrt joined
06:27
brrt joined
|
|||
harrisi | jnthn: hope you don't me bothering you, but the link to the slides for "All Your Dynamic Language Are Belong To Us (Guest Lecture at Stockholm University)" gives a 404, from your articles page (www.jnthn.net/articles.shtml). Do you have that up somewhere? It looked like it might be related to bytecode translation, and I've read all your other slides on it. | 06:29 | |
06:58
brrt joined
|
|||
samcv | good * | 07:19 | |
brrt | good * samcv | 07:21 | |
samcv | going to update Unicode Database for unicode 10 and also add in names for Hangul (Korean) codepoints | ||
since that is done algorithmically. though i still have no clue why unicode didn't just put the damn names in the unicode data file... | 07:22 | ||
there's literally no reason i can think of why they can't have just written them in there | |||
brrt | :-) | 07:27 | |
ask them! | |||
08:01
zakharyas joined
|
|||
harrisi | jnthn: I have a lot more questions and comments, but I wanted to say that your paper now has one of my favorite quotes in any research paper I've ever seen. That quote being "As in real life, getting too intimate leads to excessive complexity." :) | 08:07 | |
08:17
vendethiel joined
|
|||
jnthn | morning #moarvm o/ | 09:00 | |
harrisi: Yes, I'm that Jonathan. It's been years since I worked on bytecode translation though. | 09:01 | ||
I'm...not sure about those missing slides. Hmm. But I don't think that talk actually was about the bytecode translation work. | 09:02 | ||
Pretty sure it was about the idea of a common AST to compile different languages to. | 09:03 | ||
brrt | good hi jnthn | 09:13 | |
jnthn | o/ brrt | ||
How'd the talk go yesterday? | |||
brrt | oh, my talk is next week :-) | ||
on friday | 09:14 | ||
sorry for the misunderstanding | |||
anyway, that's basically what the GraalVM folks are doing nowadays | |||
jnthn | Oh! | ||
brrt | (a common AST) | ||
jnthn | So at TPC in Amsterdam? | 09:15 | |
brrt | uhuh | ||
i'm a bit worried that i've been 'stuck' in this subject matter for so long that what is really obvious to me isn't to others, and that i have no idea of what that is | |||
moritz | the "curse of knowledge" | 09:20 | |
jnthn | brrt: Are you expecting to merge "this week"? | 09:21 | |
brrt | ehm. maybe. i think i've uncovered another bug :-) | ||
jnthn | ah | ||
brrt | i'm not 100% sure it's not a template bug | 09:22 | |
otherwise, yeah, this week would be good | |||
i'll do a pull-request | |||
if it causes a lot of fallout, we can always switch to an enable-on-demand model for the time being | 09:23 | ||
nwc10 | brrt: building even-moar-jit with ASAN. Into NQP and getting a lot of | 09:29 | |
src/jit/linear_scan.c:433:32: runtime error: shift exponent 60 is too large for 32-bit type 'int' | |||
src/jit/linear_scan.c:429:26: runtime error: shift exponent 60 is too large for 32-bit type 'int' | |||
src/jit/linear_scan.c:437:27: runtime error: shift exponent 63 is too large for 32-bit type 'int' | |||
09:30
dogbert17 joined
|
|||
brrt | really? we shouldn't be doing that on an 32 bit int | 09:30 | |
thanks! | |||
the constant 1 must be the thing that's intepreted as an int | 09:32 | ||
that's silly | |||
nwc10 | that's C | ||
(not looked at the code, but my innacurate/incomplete understanding of C is that the constant 1 has value 1 and type int) | |||
09:35
AlexDani` joined
|
|||
dogbert17 | jnthn: is the new MoarVM, i.e. master, still supposed to be slower than e.g. 2017.07 when running multithreaded apps? | 09:48 | |
jnthn | Well, the main remaining slowdown is independent of that. | 09:51 | |
(threads or not, I mean) | 09:53 | ||
dogbert17 | I have a script which is ~20% slower in mt mode but only 10% slower in st mode compared to 2017.07 (in 32 bit mode :) gist.github.com/dogbert17/fc516b36...98b635dae4 | 09:59 | |
10:00
buggable joined
|
|||
dogbert17 | I have added callgrind results to the gist (MoarVM master) | 10:09 | |
jnthn | Yeah, a lot of time in multi-dispatch which is what I'm currently repairing/improving the spesh-time resolution of | 10:18 | |
dogbert17 | cool, I could do another run when those fixes show up | 10:20 | |
jnthn | Yeah, was working on it, then realized I need to do a bit of work on deopt | 10:24 | |
brrt | nwc10, thanks for that | 10:42 | |
today i learned, C stdint has macros INT64_C(...) | |||
and UNINT64_C(..) | 10:43 | ||
Geth | MoarVM: 0585b8d49c | (Jonathan Worthington)++ | 10 files Store deopt one target directly in bytecode. Rather than needing to use the offsets to resolve them each deopt. This brings the interpreted approach in line with what the JIT does, but further to that it means that we can have a sequence of guards that will deopt to the same place, which will be used for upcoming improvements to invocation specialization. |
10:46 | |
brrt | i think i've found my bug | 10:50 | |
it's related to the 'optimistic' store insertion | |||
Geth | MoarVM: a2128f15ba | (Jonathan Worthington)++ | src/jit/graph.c Ensure JIT uses index same as interpreter. |
10:51 | |
brrt | wait, you made the deopt target an instruction parameter? | 10:55 | |
Geth | MoarVM: 6ce4a2d1dc | (Jonathan Worthington)++ | src/spesh/optimize.h Allow calls with more args to be optimized. 4 is a little low, given nameds can need a couple of slots. |
||
jnthn | brrt: Yeah | ||
brrt | jnthn++, then jnthn++, then jnth++ | ||
jnthn++ | |||
jnthn | brrt: So could probably simplify the JIT code further | ||
brrt | oh man | 10:56 | |
i wonder why i never did that myself | |||
but anyway | |||
thanks! | |||
that helps a hole bundle | |||
*whole | |||
jnthn | The reason I want to do this is that it will let me, hopefully, have multiple guards that deopt to the same point | 10:57 | |
brrt | that makes a bunch of sense | 10:58 | |
it also is much simpler, i think | |||
because it avoids a lookup at guard-failure | 10:59 | ||
jnthn | Need to think carefully about this though | ||
'cus where we put the guards matters for what of the call resolution we can eliminate | 11:00 | ||
I was pondering stacking guards before prepargs | |||
brrt | uhuh | ||
the careful thinking is probably why i didn't do it | |||
:-P | |||
jnthn | But if I do that and they fail then we need to keep the callee stored in a register | 11:01 | |
So the deopt can find it | |||
We can still have optimized the resolution of it I guess | |||
Anyway, I guess I'll stack them up ahead of the prepargs | 11:05 | ||
Though deopt one points are presently always *after* the instruction | |||
And I'd like them before in this case I guess | 11:06 | ||
Maybe I should just spot this in the graph and handle it specially | |||
I figure in most cases prepargs will have a previous instruction | 11:07 | ||
brrt | always, but not always within the same basic block | 11:08 | |
jnthn | Yeah, hmm | 11:09 | |
brrt | (i've debugged my issue though) | 11:10 | |
jnthn | I guess I could introduce a :predeoptonepoint or so | ||
That records the offset before the prepargs | |||
brrt | (retrospectively, inc_i/dec_i were the wrong way to solve an understandable problem) | 11:12 | |
harrisi | jnthn: is there still source somewhere? | 11:21 | |
jnthn: I'm only about half way through your dissertation (4:22 am here - need to sleep), but there are a couple of things I'm not sure are covered in that, that I am interested in. | 11:22 | ||
one, stack -> register has been a complicated thing to think through, which your paper is helping with. I planned on doing stack <-> stack (.pyc <-> .class, for example), but there are a lot of type issues I can imagine | 11:24 | ||
jnthn | Good question on the source. I...actually don't know. I think I open sourced it after I was done with the dissertation | 11:25 | |
It's been > 10 years by now :) | |||
In that time the Parrot project also ceased to be developed | 11:26 | ||
harrisi | the above leads me to major question number two. if I'm going from python bytecode to java bytecode (.pyc -> .class), I have to handle types, since python bytecode is untyped and java bytecode is typed. I actually just got to the part in your paper about static analysis of .net instructions for handling stack types so I think looking more into the CLR in general could help. I looked a bit at the .pyc files of two python files that add integers and one that a | 11:27 | |
jnthn | (got cut off at "one that ad") | ||
harrisi | jnthn: yeah, the slides say it's available in the parrot source but I think I'd have to look historically. | ||
jnthn | I was going from static -> dynamic, which is an easier problem ;) | ||
harrisi | one that adds integers and one that adds strings. identical instructions, but the metadata should provide everything I need, I hope. | 11:28 | |
yeah, static -> dynamic seems somewhat easier, but I don't know much of anything. :) | 11:29 | ||
all I know is that your paper and slides have been helpful recently, so thank you. | |||
jnthn | I couldn't imagine when I did it that people would care 10 years later :-) | ||
Glad it's of some use | |||
harrisi | one more question before I go. why did you end up devoting time to parrot and other things instead of continuing with bytecode translation? | 11:30 | |
was there something that seemed like a significant show-stopper? | |||
I'm sure it's in the paper but it's hard to wait! | 11:31 | ||
Geth | MoarVM: 0ad9ce1412 | (Jonathan Worthington)++ | 5 files Make prepargs a pre-deopt-one point. Which is a deopt point where we record the offset before the instruction, so if we depot we will re-run that instruction. |
||
jnthn | Well, I got into working on Parrot because I wanted to contribute something for the Perl 6 project, and that was the most practical place I could see to help with at the time. | 11:32 | |
The bytecode translation stuff was an intresting dissertation project. If there'd been people using it, I'd probably have kept going. But Parrot never really got the up-take. | 11:34 | ||
harrisi | I'm a little over a year into community college studying computer science (with a little bit of self teaching prior), but my dreams for various languages' bytecode to be translated between each other as needed seems.. very useful, and mostly very interesting. your work is actually the first I've seen about it which talks about this at all in detail. it's a little surprising. | ||
jnthn | There has been work on this in the .Net/JVM space I believe | 11:35 | |
ut I think | |||
*But I think part of the practical problem is that most VMs get instruction sets designed for their target language | 11:36 | ||
harrisi | and there are projects like jython and ironruby but they're not bytecode translators, as far as I can tell. | ||
jnthn | And not just that, but standard library functions too | ||
harrisi | right. one of the most concerning things to me now is (thanks to your slides) the very same code behaving very different across languages. | 11:37 | |
jnthn | So at some point you reach I/O, for example, or concurrency primitives. Generally, languages do some amount of this stuff in their standard library, and then have the low-level bits call into VM-provided functions | 11:38 | |
And those will differ too | |||
harrisi | in one of your examples you used incrementing strings in python and javascript, I believe. being able to translate semantics easily and effectively seems very powerful though, and I suspect dealing with it at the bytecode level would be easier than source. | ||
jnthn | Yes, in that the semantics of bytecode are far simpler | ||
harrisi | it definitely is way out of my league, but I've been interested for awhile. :) | 11:39 | |
right. | |||
jnthn | Lunch time for me; bbiab | ||
11:40
markmont joined
|
|||
harrisi | jnthn: anyway, thank you, sincerely, for the work you did. if you for some reason you know more resources about this stuff that you could point me to, I'd be forever indebted! | 11:40 | |
night, have a good lunch! | 11:41 | ||
brrt | harrisi: the general rule is that the lower-level the 'language', the simpler the semantics, and the easier the translation | 11:46 | |
you can translate everything to everything, it is just a lot of work, and you generally lose efficiency of representation | |||
11:56
AlexDaniel joined,
vendethiel joined
11:57
travis-ci joined
|
|||
travis-ci | MoarVM build failed. Jonathan Worthington 'Make prepargs a pre-deopt-one point. | 11:57 | |
travis-ci.org/MoarVM/MoarVM/builds/260157705 github.com/MoarVM/MoarVM/compare/6...d9ce1412f4 | |||
11:57
travis-ci left
|
|||
harrisi | brrt: do you have any resources for that kind of thing for me? :) | 12:02 | |
brrt | no, that's just a general observation… :-) | 12:04 | |
you might be interested in the graal project | |||
which does something similar today | |||
well, somewhat similar anyway | |||
jnthn | harrisi: One thing that occurred to me over lunch about your mention of the python add instruction earlier: on today's JVM it probably best translates into an invokedynamic than then decides what the right thing to do is. | 12:12 | |
huh, it looks successful then says at the end the build exited with 1? travis-ci.org/MoarVM/MoarVM/jobs/260157712 | 12:16 | ||
lizmat | jnthn: fwiw, I've seen that a few times in travis reports of rakudo builds as well | 12:17 | |
not sure what to make of that | |||
jnthn | Me either | ||
12:17
AlexDaniel joined
12:41
AlexDaniel joined
12:42
zakharyas joined,
AlexDaniel joined
13:01
Geth_ joined
13:08
zakharyas joined
|
|||
jnthn | eek, language class time...making progress on call opts though:) | 13:26 | |
13:37
brrt joined
13:49
brrt1 joined
14:08
brrt joined
14:41
zakharyas joined
14:50
zakharyas joined
|
|||
jnthn back | 14:58 | ||
dogbert17 | VĂtej zp?t | 15:00 | |
jnthn | Diky | ||
lizmat | proost! | ||
jnthn | Je mi horko a musim pracovat na deopt a inlining... :P | 15:01 | |
dogbert17 | your language classes seems to be working | ||
jnthn | (It's hot and a have to work on deopt and inlining) | ||
dogbert17 had to use google translate myself :) | |||
jnthn | So it turns out that yes I can sneak extra guards in for args based on observed type tuples | 15:02 | |
And it'll net us more spesh calls and inlines (good) | |||
But sneaking them in without an entry for them in the deopt table goes rotten when we do inlining | |||
So I guess I can't cheat quite that hard. | |||
The good news is that growing that table is probably completely safe | 15:03 | ||
15:12
vendethiel joined
15:18
vendethiel- joined
|
|||
markmont | timotimo: A quick recap to be sure we're working on the same problem: | 15:25 | |
On Fedora 26, I cloned MoarVM, updated dyncall to the latest head, added a call to strerror() to src/jit/compile.c, and then ran examples/hello_world.nqp from the NQP repo: | |||
sudo setsebool selinuxuser_execstack=off deny_execmem=on | |||
env MVM_JIT_LOG=/dev/tty nqp-m hello_world.nqp | |||
This resulted in: "Setting jit page executable failed or was denied: Permission denied. Deactivating jit." together with an SELinux denial for execmem. | |||
Is this the same problem you've been looking at? | |||
timotimo | yeah | 15:26 | |
markmont | Excellent. And please let me know if I should move discussion to a mailing list to make it easier to follow. :) | ||
For dyncall, the mmap()/mprotect() code is only called by src/core/nativecall_dyncall.c:unmarshal_callback(). This code is not exercised in the hello_world.nqp example, so we don't know if the dyncall code works when execmem is prohibited (I suspect it does not work -- see below). | |||
I've looked at the JIT in PCRE2 and the one in Google Chrome V8. I've found that they both require execmem. PCRE2 does not attempt W^X at all, while V8 does use mmap()/mprotect() with the same results we're seeing in MoarVM. | |||
I've verified that the mmap()/mprotect() approach will not work with SELinux and deny_execmem. I tested under both Fedora 26 as well as under RHEL 6 (roughly equivalent to Fedora 12). Note that deny_execmem is off by default, so mmap()/mprotect() will work on most Linux machines, including those which disallow executable heaps and executable stacks. I've taken a quick look at the Linux kernel source, but haven't found an obvious place with the EPERM | |||
we're getting is being generated; I can look into this more, if desired. | |||
timotimo | you should be able to "make test" in rakudo to get the nativecall tests running | ||
15:27
AlexDaniel joined
|
|||
markmont | Yes, I've done that. But for the specific case above, I'm trying to keep things as low-level as possible. | 15:27 | |
Here is a test case showing that there is no problem with MoarVM's implementation of the JIT: gist.github.com/markmont/dcd20d632...b3bb3711ec | |||
The temporary file approach to W^X does work in all cases, see the test case at gist.github.com/markmont/b5f1ee6b4...3aae96e3b9 | |||
libffi takes the temporary file approach. It tries a number of different locations for the temporary file, searches for filesystems that are not noexec, and also attempts to create the file using O_TMPFILE so that it never appears in the filesystem and does not need to be unlinked. See github.com/libffi/libffi/blob/6e2e...res.c#L615 | |||
Based on this, I think we should decide if making MoarVM work on systems that enforce W^X in all situations is something we want to do. I think it is worth doing and am willing to work on this, but a lot of other software projects seem to be fine with the current mmap()/mprotect() approach. If we do want to force this, I think re-implementing the approach that libffi uses (and crediting them for it) would be a good way to proceed. Thoughts? (Whew, | 15:28 | ||
that was a lot of stuff!) | |||
timotimo | hm, we could potentially expose the more complicated W^X thing through a configure flag and have ifdefs for that and some .c files tat are only in there if that flag is set or something | 15:29 | |
then we don't have to do the whole searching filesystems song&dance on all platforms | 15:30 | ||
anyway, i've gotta go AFK for a bit before i can continue discussing | |||
markmont | Sure thing! Thanks for your time on this. | ||
timotimo | no, thank you! | ||
markmont | I'll hold off on proceeding until we've discussed it more. | 15:31 | |
Geth | MoarVM: 0f94262d7a | (Jonathan Worthington)++ | 2 files Widen scope of deopt annotations function. And make it a tad more generic. |
15:39 | |
15:49
robertle joined
|
|||
Geth | MoarVM/spesh-invoke: 290e2545be | (Jonathan Worthington)++ | 2 files Use callsite info to insert guards before prepargs These will allow us to use the type tuple to both pick a spesh candidate that already exists as well as do multi-dispatch, even in the presence of args passedin Scalar containers. This commit only adds the guards, and doesn't yet make use of them. |
16:02 | |
16:03
brrt joined
|
|||
jnthn | Phew :) | 16:04 | |
16:07
zakharyas joined
|
|||
brrt | \o | 16:12 | |
Zoffix | | | 16:13 | |
/\ | |||
brrt | i've tracked down the errors in the JIT to the use of too-small variants for the bitmap ints | 16:14 | |
16:15
travis-ci joined
|
|||
jnthn | haha, spot the bug in the above commit :) | 16:15 | |
travis-ci | MoarVM build errored. Jonathan Worthington 'Widen scope of deopt annotations function. | ||
travis-ci.org/MoarVM/MoarVM/builds/260255405 github.com/MoarVM/MoarVM/compare/0...94262d7a29 | |||
16:15
travis-ci left
|
|||
jnthn | heh, not you travis :P | 16:15 | |
Zoffix | :D | 16:16 | |
brrt | it's not totally obvious initally | 16:17 | |
markmont: if you do want to do this, please be my guest | |||
Zoffix | there's too much code for bug spotting :) | 16:18 | |
markmont | brrt: I'd like to look into this more. I'm sure timotimo will have more input when he gets back. | 16:19 | |
jnthn | Turns out I didn't bump the usage count for the register it deconts into, so the guard was against an empty register and so always failed | ||
brrt | my personal opinion (which is of no importance) is that W^X that can be circumvented by a filesystem hack is security theater | 16:20 | |
and that people who insist on employing can enjoy the fulll benefits of doing so | |||
amongst which there are, no functional JIT | |||
markmont | Yeah, the Mozilla devs have a bug report on this from back in 2006 that says much the same thing. | ||
brrt | now, if we were to go so far as make a full ELF and DLL, then load that via dlopen, then use that, well, i'd be more appreciative, cause i can see that having some real benefits | 16:22 | |
markmont | It's not clear to me why SELinux denies the mprotect() call. The description of the deny_execmem boolean says "Deny user domains applications to map a memory region as both executable and writable", but we're not trying to do that. | ||
16:23
zakharyas joined
|
|||
brrt | hmm, i think the question is if the currrent sequence of call adequatly makes the page unwritable | 16:23 | |
and, of course, if the selinux check is correct | |||
markmont | I agree that that question is still open. I've looked at several other JITs, though, and none of them do any better. | 16:25 | |
In other words, I'm not seeing anything obvious that is missing in terms of making the page unwritable. | |||
brrt | anyway. i'm personally not super-fond of 'hacks' like searching for non-noexec filesystems. that feels to me like introducing less security (because the filesystem can be overwritten), not more | 16:27 | |
but like i said, you're free to try it out | |||
and i'll be interested to see what the result will be :-) | 16:28 | ||
Geth | MoarVM/spesh-invoke: ddb3e65596 | (Jonathan Worthington)++ | src/spesh/optimize.c Make sure to mark decont for guard used. Otherwise, dead instruction elimination throws it out and we guard it against an empty register, which will always fail. |
16:29 | |
MoarVM/spesh-invoke: 87fb83fc9a | (Jonathan Worthington)++ | src/spesh/optimize.c Guards need STable, not type object. |
|||
MoarVM/spesh-invoke: c47281fb40 | (Jonathan Worthington)++ | src/spesh/optimize.c Use type tuple to find spesh candidate if present. If not, we fall back to using the facts inferred using the facts established thus far. |
16:30 | ||
markmont | brrt: OK, thanks. I agree with you about hacks leading to less security. I don't know if there is really anything to fix here, but I'd still like to look into exactly why the mprotect() returns with EPERM since we're asking to remove PROT_WRITE and add PROT_EXEC. | 16:35 | |
Geth | MoarVM/even-moar-jit: 8 commits pushed by (Bart Wiegmans)++ | 16:37 | |
brrt | yeah, thanks for looking into it | 16:38 | |
okay, afk | |||
jnthn | Good news is the above pass spectest :) | 16:52 | |
Uh, my ones :) | 16:53 | ||
Hopefully brrt++'s too :) | |||
Geth | MoarVM/spesh-invoke: e7f7a18e2e | (Jonathan Worthington)++ | 3 files Enable type tuple use in multi-dispatch. Should get Perl 6 multi-dispatch resolution at spesh time mostly working again. |
17:23 | |
jnthn | Enough! | 17:26 | |
bbl | |||
18:36
AlexDaniel joined
|
|||
nwc10 | jnthn: ASAN doesn't find your branch at all interesting | 19:27 | |
jnthn | Nice :) | 19:43 | |
Hm, I think somewhere there's an RT about a spesh bug involving native args | |||
Think I just found what's going on after running into it in some other code | 19:44 | ||
In an inline, this: | |||
arg_i liti16(0), r9(5) | |||
arg_i liti16(1), r15(1) | |||
Is rewritten to: | |||
set r30(1), r9(5) | |||
set r30(2), r15(1) | |||
It puts two different args into the same register o.O | |||
ooh, shop closing soon; brb | 19:45 | ||
nwc10 | .tell brrt ASAN finds even-moar-jit very exciting: paste.scsys.co.uk/564761 | ||
yoleaux | nwc10: I'll pass your message to brrt. | ||
jnthn | OK, figured out what it is. Sometimes the thing we're inlining grabs the parameters and puts them into different registers | 20:17 | |
uh, into the same registe | |||
*register | |||
But does something (like boxing them) inbetween. | |||
Which is fine enough | |||
Until we inline | |||
nwc10 | .tell brrt first bad commit is 2dbb62f9cade2, "Add sp_decont op" - as the only non comment change in that is to add a template, I guess it reveals a latent bug | 20:28 | |
yoleaux | nwc10: I'll pass your message to brrt. | ||
Geth | MoarVM: 3f33a8419b | (Jonathan Worthington)++ | src/spesh/inline.c When inlining, replace receive instruction. If two parameters are received into the same register (because, for example, they are then boxed and only the target of the box matters), then the arg passing code would be rewritten in such a way that the second argument overwrote the value of the first. This resolves the issue by flipping the rewriting to the receiving side, where this issue cannot happen. |
20:38 | |
20:58
DBeepBeep joined
|
|||
lizmat | jnthn: reality check: sub a(Any:U $a) { say nqp::isconcrete($a) } # can that ever say 1 ?? Don't think so, right ? | 21:00 | |
jnthn | No | 21:02 | |
:U is defiend in terms of isconcrete | |||
lizmat | ok, figured, thanks! | ||
21:03
https_GK1wmSU joined
21:04
https_GK1wmSU left
|
|||
lizmat | argh, I thought I could simplify code, but I can't after all... will add comment just someone doesn't step into that trap again later | 21:05 | |
21:26
travis-ci joined
|
|||
travis-ci | MoarVM build passed. Jonathan Worthington 'When inlining, replace receive instruction. | 21:26 | |
travis-ci.org/MoarVM/MoarVM/builds/260362737 github.com/MoarVM/MoarVM/compare/0...33a8419bfd | |||
21:26
travis-ci left
|
|||
markmont | brrt, timotimo: A W^X update: | 21:29 | |
SELinux not allowing mprotect() to change from PROT_WRITE to PROT_EXEC turns out to be deliberate in order to avoid exploiting the writable memory with a timing attack in a multithreaded program as described in | 21:30 | ||
www.internetsociety.org/sites/defa...09_2_2.pdf | |||
It turns out that the libffi implementation (trying different locations, searching for noexec filesystems, using O_TMPFILE) originated in Mozilla Firefox nanojit, | |||
timotimo | mhm mhm | ||
markmont | see bugzilla.mozilla.org/show_bug.cgi?id=506693 (lots of good argument about temporary files there). | ||
see bugzilla.mozilla.org/show_bug.cgi?id=506693 (lots of good argument about temporary files this). However, OpenBSD 6.1 *will* allow it. | 21:31 | ||
Whoops, that didn't come out right, let's try again: | |||
NetBSD 8.0_BETA won't allow mprotect() to change from PROT_WRITE to PROT_EXEC by default (tested and verified). But OpenBSD 6.1 will. | 21:32 | ||
Under NetBSD 8.0 it's possible to mark individual executables specially to allow them to use mprotect() this way, overriding the default behavior. | 21:33 | ||
timotimo | i'm going to read that paper | 21:34 | |
markmont | So the bottom line is that people on systems that allow it say "use mmap() and mprotect()" while NetBSD people and Linux people who use deny_execmem say "map a temporary file twice". | ||
MasterDuke | timotimo: do you have a fix for the profiler segvs in the works? or do you need any more info? | 21:37 | |
timotimo | MasterDuke: haven't looked at it yet, tbh. i was only really aware of the "can't call defined on a null method" thing | 21:40 | |
MasterDuke | i don't know if there are multiple reasons | ||
timotimo | "on a null object"* | 21:42 | |
that one's a spesh-related problem (i.e. likely optimizer trouble) | |||
MasterDuke | ah, this line?: STABLE(code)->invoke(tc, code, cur_callsite, args); | 21:43 | |
that's what valgrind pointed to | |||
timotimo | markmont: our jit differs from what is described in that paper because we never make pages writable after making them executable | 21:45 | |
markmont | indeed, but it's disallowed anyway. There's some more information about this in the Mozilla bug above. | 21:47 | |
timotimo | OK, i shall read that after i've finished with the paper | 21:49 | |
MasterDuke | jnthn: seems like --profile breaking might be due to your recent spesh work, any ideas? | 21:50 | |
timotimo | markmont: the drawback to what we're doing is that we're wasting at least one whole page for even the smallest pieces of jitted code | 21:53 | |
so at some point we might want to re-use memory that was once execute-only and add some more jitcode at the end | 21:54 | ||
markmont | Calling mprotect() again to switch back should work on platforms that support it, if we stick with the current approach. Or if we go with the map-a-temp-file-twice approach, that can be reused without doing anything (but we'll consume a file descriptor for each one we have). | 21:56 | |
timotimo | ah, yes, we'd reach the max number of open files in no time at all | ||
markmont | I imagine it might be possible to keep many pieces of jitted code in a single mapped-but-unlinked tempfile. I'd have to try it. | 21:58 | |
In other words, map per interpreter or once per thread rather than once per piece of jitted code. | 21:59 | ||
timotimo | if we have per-thread code pages we'll be regenerating many of the same jitted blobs on all threads | 22:01 | |
markmont | right -- one per interpreter/VM then, or whatever the correct number turns out to be. I'm very new to MoarVM and NQP and not familiar with how they work. | 22:04 | |
timotimo | we don't do any of the jit-related mitigation techniques discussed in that paper | 22:07 | |
like making the jitted code slightly random and unpredictable | |||
jnthn | MasterDuke: No immediate guesses, sorry. If nobody else manages to hunt it down, I'll get there, just got quite a queue of spesh things to do. | 22:17 | |
timotimo | at some point hopefully a Real Security Researcher™ will try to attack moar | 22:19 | |
i imagine if you can reach the nativecall functions, you'll be able to wreak absolute havoc | |||
i'm already unreasonably tired ... but maybe i'll go to bed rather soon today | 22:21 | ||
22:24
jsimonet joined
22:47
https_GK1wmSU joined
22:49
https_GK1wmSU left
22:53
markmont joined
|
|||
jnthn | Oh heck. It turns out that we don't get UV_EOF in the socket callback when the socket is closed | 23:17 | |
So never pass on the done signal | |||
And so leak | |||
23:23
Geth joined
|
|||
MasterDuke | jnthn: the --profile problem is interp.c:973, but that's because of github.com/MoarVM/MoarVM/blob/mast...me.c#L1764 | 23:56 | |
if i add a `if (!code) fprintf(stderr, "here!\n")` right before then it gets printed | 23:57 |