00:00 MasterDuke joined, ChanServ joined 00:12 ilmari[m] joined 01:53 ilbot3 joined
Geth MoarVM: markmont++ created pull request #643:
Add W->!X JIT support for NetBSD 8, SELinux with deny_execmem, PaX		 02:28
03:16 pharv_ joined
MasterDuke would anybody mind merging github.com/MoarVM/MoarVM/pull/626 if there aren't any questions/concerns? it'd be nice to get it in so i don't have to keep updating for new oplist changes and force pushing. thanks 03:33
samcv looks 03:53
hmm those strlen calls colud probably be turned into string[0] != NULL though that's just what i would have done. since it's only called if the env var is set, the performance nick is not really an issue 03:58
though you can just shorten that to string[0] and make it cleaner 04:00
then you don't have to make the strlen calls
geekosaur <nit> NULL is a pointer, and on some OSes this is enforced </nit> 04:01
(that is' you can't assume it will work as an (int) or (char) or etc.) 04:02
samcv yeah you don't need to put NULL
you can just have it eval using integer logic
if (string[0]) works fine
and where you want to check if the length is 0, if (!string[0]) 04:04
geekosaur, also thanks for the tip about that 04:05
geekosaur well, it's more of a portability gotcha
samcv char is essentially integer based anyway so only should do NULL if it's a pointer
geekosaur on many systems it's just #define NULL 0 and that works
samcv though a compilier may do what you mean
geekosaur but a very few to #define NULL ((void *) 0)]
...typoes
but a very few do #define NULL ((void *) 0) 04:06
(hypothetically someone could use a const as well.)
04:08 hoelzro joined
samcv i'm going to change all the unneeded strlen in the file. all the ones we only care if == 0 or != 0 and don't need the actual value 04:09
geekosaur, do you think we want to print to stderr when we have certain variables enabled, like SPESH log on or whatever 04:16
print to stderr that we are outputting to stuff to "XYZfile.txt" or whatever
Geth MoarVM: 911b6323de | (Samantha McVey)++ | src/moar.c
Remove unneeded strlen() calls in moar.c

When we are only checking `if (strlen == 0)` or `if (strlen != 0)` we can instead replace this with `if (string[0])` or `if (!string[0])` and avoid unneeded function calls.		 04:17
samcv MasterDuke, the travis build failed
04:18 timotimo joined
samcv MasterDuke, finished my review 04:20
05:13 pharv_ joined 05:46 brrt joined
brrt . 05:48
yoleaux 14 Aug 2017 18:13Z <samcv> brrt: so the main thing to see there that it read 1 byte too much? looks like it's coming from threebyte_memmem since you use os x so it uses our 3rdparty/freebsd/memmem.c file
14 Aug 2017 18:14Z <samcv> brrt: is that reproducable? can you tell me how you produced it? i can manually use the freebsd memmem/nonfreebsd memem and see if any diff
brrt .tell samcv that it is reproducable, yes, but the reproducing test case is kind of large 05:50
yoleaux brrt: I'll pass your message to samcv.
brrt i'll try and golf it
i'm expecting it is partially a concurrency error for the application 05:51
06:08 mtj_ joined 06:23 brrt joined 06:35 brrt joined 07:01 brrt joined
samcv THANKS :) 07:08
yoleaux 05:50Z <brrt> samcv: that it is reproducable, yes, but the reproducing test case is kind of large
samcv oops caps
brrt hmmm 07:10
i'm having all sorts of concurrency errors :-(
08:34 domidumont joined
jnthn morning o/ 08:49
brrt moarning 08:58
09:12 domidumont joined
brrt did you see the broken-concurrency-in-uzu test case 09:13
i'm not 100% positive it's not just uzu holding concurrency wrong 09:14
jnthn Not yet; is there an easy reproduction? 09:16
09:23 zakharyas joined
Geth MoarVM/atomics: 56df24161d | (Jonathan Worthington)++ | src/core/interp.c
Implement barrierfull op.
 09:31
MoarVM/atomics: 56421faef2 | (Jonathan Worthington)++ | 2 files
Add a way to size a P6int to the atomic size.
MoarVM/atomics: af833439d5 | (Jonathan Worthington)++ | 2 files
Add a way to size a P6int to the atomic size.
 09:46
brrt yeah, if you check out the test suite at a specific commit, it will die in all sorts of interesting and more importantly inconsistent ways 10:06
ilmari keeps reading P6int as Pint 10:17
jnthn Mmmm 10:27
Bit warm outside for me to fancy pub lunch today though :)
10:50 markmont joined 10:58 brrt joined
Geth MoarVM/atomics: e6ab1db5ff | (Jonathan Worthington)++ | 5 files
Implement atomic load/store of native int lex refs
 11:08
11:26 harrow joined
markmont Is there anything I can do to help get the version of dyncall updated? github.com/MoarVM/dyncall.git would have to be updated from dyncall's upstream repo, which is hg. jnthn and timotimo thought that bumping the version would be fine in the discussion at irclog.perlgeek.de/moarvm/2017-08-01#i_14953751 11:33
This would get rid of executable stacks and make MoarVM easier to use on hardended systems. 11:34
jnthn markmont: If you submit a PR against the repo you just mentioned that does the update, I'd happily merge it 12:12
markmont OK, thanks! I'll get started on that.
Geth MoarVM/master: 4 commits pushed by (Mark Montague)++, (Jonathan Worthington)++ 12:13
12:14 brrt joined, buggable joined 12:15 Geth_ joined 12:16 ZofBot joined
Geth MoarVM/atomics: 9c6de1ae8c | (Jonathan Worthington)++ | 3 files
Atomic integer inc/dec/add.

Only on lexicals until handling of other native reference types is implemented.		 12:23
brrt i'd be interested to go to the vm meetup as well
but i feel a bit like it might be over my league?
jnthn brrt: Heh, that's just how I feel. "Is my work really interesting enough for it?" :) 12:24
But I guess if I'm being encouraged to go by I guess a regular attendee of it on Twitter, then presumably it is. :)
brrt well, i'm not on twitter, and wasn't encouraged, so that doesn't count for me :-) 12:25
but yeah, you should totally go
jnthn fwiw, I don't think it'd be at all out of your league 12:26
Plus it'd be interesting to have another MoarVM hacker there to discuss what ideas from other talks we might take inspiration from :) 12:27
lizmat and who doesn't want to be in Prague? :-) 12:49
12:53 brrt joined
jnthn Me, when the temperature here gets over 30C :P 12:55
Zoffix Is air conditioning not a common thing in Europe? 12:57
jnthn Depends where you are in Europe. But here, no.
Zoffix Here, I walk from air conditioned apartment, to an air conditioned bus, to an air conditioned office. It could be 60C, for all I care :)
jnthn There just aren't that many days of the year when it's that hot.
And if I didn't have a south-facing office and apartment I'd probably care less also :) 12:58
Pretty sure the trams are air-conditioned. :)
Well, OK 12:59
Pretty sure the *modern* trams are air-conditioned.
The quaint old ones ain't
Geth MoarVM/atomics: af17527147 | (Jonathan Worthington)++ | 3 files
Implement cas_i op.
 13:01
jnthn brrt: github.com/MoarVM/MoarVM/pull/643 looks pretty sane to me, but if you get a moment to glance over it that'd probably be good :) 13:06
brrt oh, had not seen it yet 13:11
i'll take a look sure
13:50 brrt joined 14:00 brrt joined
brrt hmmmf 14:01
i've looked at it
i don't like it very much
it's probably right, and it probably serves a purpose
i don't like the compile-time split between having libffi and not having it 14:02
markmont++ for making it though
markmont I don't know the history between dyncall and libffi in MoarVM, but it seems to me that libffi has more features than what gets used by default, which is dyncall. One way to avoid the complie-time split is to just get rid of dyncall and bundle libffi as a 3rdparty submodule -- but this is a fairly radical change and I'm not proposing it. 14:05
brrt also, i'm stil in denial about the need for this patch 14:08
:-P
i may be wrong about that for sure
14:12 zakharyas joined
Geth MoarVM/atomics: 175d63f79e | (Jonathan Worthington)++ | 9 files
Enable native int atomics on P6opaque attributes.

Provided they are ints of the machine's atomic operation size.		 14:18
markmont brrt: No, your points are good ones. From my perspective, this is just about making Rakudo Perl 6 run on hardened platforms without the person who installs it jumping through difficult, platform-specific hoops and potentially giving up and using some other language instead. 14:20
brrt yeah, i can see that point as well
jnthn So far as I remember, libffi is a pain to build on Windows, so dyncall is at least preferable there
brrt but windows doesn't do W^X 14:21
yet
markmont On NetBSD 8, the choice is between "build Rakudo with --moar-option='--has-libffi'" and "be sure the installer runs paxctl -M .../perl6/bin/moar before building NQP"
jnthn brrt: Yes, I'm explaining why I'd not care to drop dyncall :)
brrt to be really honest, i don't see the problem with adding that to our build system 14:22
(the paxctl ....)
markmont On SELinux, the alternative is "why are you enabling deny_execmem, don't do that!" / "well, write and install a local SELinux policy that runs moar in its own SELinux domain and grants it the execmem permission".
jnthn markmont: I figure the first one is easier in that it can be handled as part of packaging?
brrt also, i have (rephrasing) a 'bias to insource', why don't we do the equivalent thing ourself 14:23
matter of fact
markmont Making Rakudo work on NetBSD and SELinux with mprotect() could both be handled as a part of packaging, yes.
brrt can we not redo the API so that we get a 'swap this read/writeable pointer with a read/execable pointer'
and then do the 'difficult' thing (whatever it is that libffi does) only on platforms that we need it 14:24
markmont brrt: I considered that. We'd need to internally allocate an additional pointer or two worth of storage to store the addresses, so we can produce them on demand.
It seems more complicated, code-wise. 14:25
brrt right
but more contained, i'd argue
and considerably less weird than having two-pointers-to-the-same-thing
i mean, you could have a struct MVMExecMemory { …. }
markmont Except the two-pointers-the-the-same-thing is what the security experts are pushing, rightly or wrongly. So I was keeping it in line with what other people might expect. 14:26
brrt oh, they can push that all you like. it's just that it's not part of the interface i want to reason about 14:28
*they
i want to have a simple notion of: allocate rwx memory, get writable, get executable, destroy 14:29
also, people honestly don't expect that much :-P
markmont :)
brrt it's tricky to get people to care about memory management internals :-) 14:30
(they care more about not having written it in rust, but that is another matter)
markmont I think it will be more code and complexity to implement your notion on platforms that enforce W^X, but I'm willing to rewrite the patch that way if you like in order to get a simpler and cleaner API. 14:32
brrt on the other hand, like jnthn said, it s pretty much sane
jnthn I guess if we do it the way brrt is suggesting then we will solve it for anyone who builds with dyncall also...
markmont No, we'd have to reimplement a fairly hefty chunk of libffi in order to solve it for people who build with dyncall. We can do that, of course, if we want. 14:33
brrt hmmm 14:34
jnthn Ah, OK
brrt so i guess we have three alternatives
one
accept the patch as is
two
modify the patch to contain the changes in a single API that hides the per-platform differences 14:35
three
modify our build system to install the right flags on hardened platforms
markmont That sounds accurate.
brrt okay 14:36
jnthn Might 3 be considered a bit antisocial?
brrt why?
markmont Note that option 3 means that sudo needs to work in order to get something that functions on a hardened platform, otherwise you'll get what we have now.
jnthn Or wait, did I misunderstand it?
brrt ah, i see, that's a disadvantage, yes
jnthn I just worry that a build system that changes flags on the target users's platform might not go down too well
s/flags/security-related configuration/ 14:37
brrt they'd be localized changes to the moar binary only, i think
that's how i understood them
markmont brrt: yes
jnthn Oh, I see
OK, it's less bad
brrt well, we'll get some flames for that, for sure\
'why does your stupid binary set up these insecure flags? don't you like user security?' 14:38
markmont But, we'd be granting ourselves access that is not default. This would be documented, of course, and the owner of the system could undo or disable it if they didn't like it.
brrt correct
so it is a *bit* antisocial still
markmont This started out as a flame on the perl6-compiler mailing list. I picked it up because I run my Linux systems with SELinux and deny_execmem.
...so I was interested in improving the situation.
In other words, we already got a flame :) 14:39
brrt hehe
yeah
jnthn The flame being what?
brrt to be fully honest, i found the 'binary nqp blob' thing being more worrisome
markmont "You suck for not working with W^X"
jnthn Ah
brrt 'you suck for so many reasons', it was more like
:-P
and rakudo star didn't have signed builds 14:40
markmont Yeah, here's the message, the W^X part is near the end: www.nntp.perl.org/group/perl.perl6....16180.html
brrt oh by the way 14:46
panicking because we can't JIT something is not good
we do it in some places
we really shouldn't 14:47
panicking because the JIT is in an inconsistent state is also different from panicking because we couldn't allocate memory
we can always not JIT
(which is by the way also an acceptable way to go) 14:48
markmont I originally had that as disabling JIT. I changed it to panic, though, because if it fails it almost certainly means malloc() failed. And in that case, I thought we should panic rather than proceed and have the *next* malloc fail.
brrt but this is special malloc
anyway. thanks for the PR. I'll think about it some more 14:49
markmont Yes, although "it depends". On most platforms it will be regular malloc(). libffi will only do the mapping to a temporary file and use its special malloc if it needs to.
With that said, I'm not opposed to changing it to just disable JIT.
14:49 brrt joined 15:00 releasable6 joined 15:06 brrt joined
brrt okay, i was walking on the way to the train and another option occured to me 15:06
what if we probe our ability to get executable memory at build-time, and disable the JIT (with a message and an explanation) on those platform where it won't work 15:07
because that's option 4: disable the JIT by default for platforms that are 'hardened' in this way 15:08
they can still get the extra performance boost, but they'll have to jump through some hoops to do so
15:09 zakharyas joined
markmont brrt: If we did that, I'd say that the behavior should be to just change the default value of MVM_DISABLE_JIT=1. That way, if they jump through the extra hoops they can simply set MVM_DISABLE_JIT=0 and not have to rebuild. I personally prefer having W^X support, but your option 4 would work and be an improvement over what we have now. 15:11
brrt so, stupid question time agian 15:12
jnthn I'd prefer that we try to provide some W^X support also 15:13
brrt does luajit work on openbsd
hmmm
jnthn I'm not convinced by the security mechanism, but it's not really important whether I am or not.
brrt eh, netbsd
jnthn What's important is the experience Perl 6 users on such platforms get.
If other language's JITs fall apart then it's an oportunity for us to be better. :) 15:14
brrt :-) i'm asking specifically about luajit because i have the sources checked out and they do the same thing as we do
i'd really like to discuss with some of the experts who thought this up 15:16
because i'm just…. still confused about it all 15:17
timotimo jnthn: i'm looking into rebuilding the spesh graph after our first pass of inlining and bb elimination; do you think it's worth trying to manipulate the spesh graph into the new form or just codegen the flat representation and completely rebuild the graph from that? 15:18
jnthn I'm not sure we need to do that 15:19
I'd (1) fuse BBs with a single successor after spesh, and (2) recalculate the dominance tree
It should still be in fine SSA form 15:20
timotimo OK, rebuilding the dominance tree; would that also mean rewriting the versions of variables?
or just which BBs dominate which other BBs?
jnthn No, just which dominate which
So we can walk in order 15:21
I don't think we need to go re-establishing all the facts
I mean, since we already have the dead writer thing...
The PHI merges should be straightened out 15:22
What are you wanting to acehive by doing the rebuild of it, BTW? 15:24
markmont brrt: I took a look at the luajit v2.1 branch and agree that it doesn't support hardended systems. Google V8 does not, either. 15:25
timotimo i'm wondering if the versions get kind of out of sync by some of the optimizations i made (which tend to be a little sloppy)
jnthn timotimo: Ah. If so we should really clear than up
timotimo: That's exactly the kind of hackiness I've been trying to get rid of in spesh 15:26
timotimo probably by making the opts better :)
jnthn 'cus it reliably trips us up
timotimo in case the BBs can't actually be fused because of bb-splitting ops, should i put the BBs in the linear order they'd be in?
jnthn Normally, a little later when we manage to opt a bit more aggressively
I think we'd probably be better off just trying to arrange doing inlines "in the right place" rather than at the end 15:27
Then we'd save two gotos per inlined thing too
I don't think there's any blocker for doing it by ow 15:28
*now
It's just a case of splicing it into the linear_next correct and renumbering (the numbers are only really used for debug output anyway)
I'd probably leave the goto at return in as there can be multiple return points
15:29 zakharyas joined
timotimo ah, right 15:29
15:32 releasable6 joined 15:34 releasable6 joined
brrt markmont: does that patch actually work? 15:36
jnthn (And the last one will be remvoed for you by the opt that gets rid of unrequired gotos)
brrt speicifcally, does the free bit work
because
dasm_encode will just overwrite whatever is in 'writeable memory'
which is in this case an initialized ffi_closure object 15:37
which has pointers to the table from which it is allocated (which contains the free list)
and the pointer to *code is a part of the page that is allocated for the table 15:39
markmont As far as I can tell, yes, it all works. I did instrument jit/compile.c though and found that MVM_jit_destroy_code does not get called in any of the scenarios I tested, which include running the Rakudo spectest and doing a `zef install Bailador` with all dependencies.
brrt hmmmmm
and you don't run into cases where you overwrite the page
then something magical magic is going on for sure
jnthn Well, I guess we'd only collect JITted code if we had an EVAL that got hot :) 15:40
brrt check out the definition of ffi_closure 15:41
github.com/libffi/libffi/blob/mast....h.in#L288
and ffi_closures_alloc: github.com/libffi/libffi/blob/mast...res.c#L198
oh, wait a minute
there are multiple definitions of that function 15:42
15:42 pharv_ joined
jnthn m: my @a[2;2]; @a[1;1]++; 15:42
camelia ( no output )
brrt github.com/libffi/libffi/blob/mast...res.c#L831
this is the one which you should actually see executiong
that's.
jnthn m: my int @a[2;2]; @a[1;1]++;
camelia Cannot resolve caller postfix:<++>(Int); the following candidates?match the type but require mutable arguments:? (Mu:D $a is rw)? (Int:D $a is rw)??The following do not match for other reasons:? (Bool:D $a is rw)? (Bool:U $a is …
brrt some evil stuff
jnthn grmbl
brrt (yay bugs) 15:44
anyway, that gives me option 5
we can 3rdparty dlmalloc, just like libffi
no, never mind that suggestion 15:45
on the other hand 15:48
can't we dualmap the page as writeable ourselves
eh, executable
markmont Yes, but we have to do it by mapping to an actual unlinked file, which involves trying a lot of things. Then, we have to have a way to allocate and free memory from the mapped regions, which is what libffi uses dlmalloc for. I figured that since MoarVM already had the option to use libffi, why reinvent the wheel? 15:50
Geth MoarVM: MasterDuke17++ created pull request #644:
Fix typo		 15:53
MoarVM/atomics: 79f9cd9b17 | (Jonathan Worthington)++ | 12 files
Provide integer atomic ops on native arrays.

Both single and multi-dimensional ones. With this, all cases of atomic ops on native atomic-sized integers are done.		 16:06
16:07 brrt joined 16:14 brrt joined
Geth MoarVM/atomics: 286a67adb8 | (Jonathan Worthington)++ | src/6model/6model.h
Fix typo; MasterDuke17++.
 16:18
brrt makes some sense, yes 16:20
okay, so i guess my question is, do we really, really have to do the file dance, or can we dualmap in memory 16:23
Geth MoarVM/atomics: fa8a3c99b8 | (Jonathan Worthington)++ | src/gc/debug.c
Fix compilation error after new error added.
 16:27
markmont jnthn: I've created the pull request to update dyncall, github.com/MoarVM/dyncall/pull/4 16:31
jnthn markmont: Alright; guess you did a test build against this version and saw make test in Rakudo passed too? 16:32
markmont Yep!
`make test` and `make spectest` both all passes.
jnthn Alright, sounds good
Merged that 16:33
Home time; bbl 16:39
samcv good * 16:40
Zoffix \o 16:43
How come Geth didn't report the merge? Is "Just the push event" instead of "Send me everything" selected for the web hook? 16:45
MasterDuke samcv: hey hey hey. since you commented on it last, how does github.com/MoarVM/MoarVM/pull/626 look now?
samcv i was looking at it right now coincidentally :)
MasterDuke excellent
samcv did you revert some of the ops file changes and rerun tools/update_ops.p6 ? 16:46
MasterDuke yeah
samcv cool 16:47
Geth MoarVM: b800871248 | (Samantha McVey)++ | src/core/nativecall.c
Remove two more unneeded strlen from nativecall.c

  See github.com/MoarVM/MoarVM/commit/91...69c8857d89
for reasoning.		 17:35
MoarVM: 633cb35a2d | (Samantha McVey)++ | src/strings/ops.c
Only foldcase the needle for ignorecase operations

There was a bug where we may foldcase a needle that was only wanted for an ignoremark operation that should not have been foldcased.
jnthn Zoffix: Maybe we don't have anything set up for that repo. I think this is the first time we merged anything there in a year or so... :) 18:10
Zoffix Oh, I thought it was MoarVM/MoarVM repo. 18:13
19:04 ilbot3 joined 20:09 markmont joined, zakharyas joined 21:04 markmont joined
Geth MoarVM: MasterDuke17++ created pull request #645:
Fix what I believe is a typo in a comment		 21:09
MoarVM/master: 6 commits pushed by MasterDuke17++, (Samantha McVey)++ 21:23
MoarVM: 242771d32e | MasterDuke17++ (committed using GitHub Web editor) | src/6model/reprs/VMArray.c
Fix what I believe is a typo in a comment
 21:24
MoarVM: 1f8417ece9 | (Samantha McVey)++ (committed using GitHub Web editor) | src/6model/reprs/VMArray.c
Merge pull request #645 from MasterDuke17/patch-2

Fix what I believe is a typo in a comment
MasterDuke samcv: thanks 21:25
samcv np :) so the coverage ops will let us selectively check coverage right, that sounds pretty useful
MasterDuke yeah 21:28
i plan to add support for it to coverable
but it will allow you to see coverage of just the code you run, excluding whatever runs at startup for example 21:29
gonna add the op to nqp and bump its moar_revision 21:32
samcv ignoremark is ready for release now. added bunch of tests to roast plus new (previosly the op was untested in nqp tests) nqp tests 21:34
timotimo huh, i got back to the 'puter a *bit* later than i expected 22:26
22:30 travis-ci joined
travis-ci MoarVM build failed. Samantha McVey 'Merge pull request #626 from MasterDuke17/coveragecontrol_op 22:30
travis-ci.org/MoarVM/MoarVM/builds/264915963 github.com/MoarVM/MoarVM/compare/6...78b483cd93
22:30 travis-ci left
timotimo the bump came in while an earlier commit was being tested or something? 22:32
MasterDuke probably, there were a couple bumps 22:33
timotimo glad to see coveragecontrol be merged 22:38
MasterDuke likewise. now to update coverable... 22:42
timotimo: have you seen this before? travis-ci.org/perl6/nqp/jobs/264930803 the other jobs were fine, i'm not sure that it's actually something specific to my code 23:41
timotimo huh, that's not good 23:43
it shouldn't do that 23:44
all your code does is put some optional stuff in that doesn't even get touched during the build process
i don't think it's your code that did anything here
MasterDuke good (and bad) 23:45
timotimo problems like this one should be findable through nursery size reduction and gc debugging increase, though 23:48
MasterDuke dogbert17_: you are good at tracking this sort of thing down, if you feel so inclined there's an `MoarVM panic: Heap corruption detected: pointer 0x2ad2a40ba450 to past fromspace` hiding somewhere in the NQP build 23:50
.tell dogbert17_ irclog.perlgeek.de/moarvm/2017-08-15#i_15021638 if you're interested 23:52
yoleaux MasterDuke: I'll pass your message to dogbert17_.
23:57 pharv_ joined
Geth MoarVM: MasterDuke17++ created pull request #646:
Remove impossible check		 23:57

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!