[00:27] *** leont joined [01:23] *** colomon joined [01:34] *** geekosaur joined [01:44] *** geekosaur joined [01:46] *** geekosaur joined [02:56] *** vendethiel joined [04:18] *** geekosaur joined [06:26] *** japhb joined [07:34] good *, #moarvm [07:48] *** zakharyas joined [07:56] *** patrickz joined [08:05] *** FROGGS joined [08:23] *** zakharyas joined [08:49] *** ShimmerFairy joined [09:54] *** zakharyas joined [10:04] *** brrt joined [10:06] *** leont joined [10:06] ohhai #moarvm [10:06] i just had a mildly evil idea that i just can't find the flaw on yet [10:07] you might know that i've added a facility to extract the values used into a single tile out of the tree into a linear array; that is very useful for ensuring that values which should be in a register are actually there [10:08] as well as ensuring that values which are never further used are 'killed' [10:11] anyway, a very simple and practical difficulty in tiler linearisation is that i want to interleave the tiles with nodes representing the 'inter-tile-work' that is done by the compiler [10:11] that is to say, i want to insert loads, stores, labels, and conditional jumps, because that is all there is [10:11] bear with me; i'll get to the point :-) [10:12] these things all take integer values as arguments [10:12] i can store any number of integers in the expression tree as arguments [10:12] i can extract a number of intergers out of the expression tree, by the above-mentioned mechanism [10:13] hence, i can insert the arguments for these functions into the tree, and deal with them as i would with regular tiles [10:13] i can create fake pseudotiles, and all will be well.... [10:14] but it's conflating of the tree and its' storage mechanism, for one thing, and reusing that storage mechanism for nefarious purposes, too [10:15] hence, big software engineering red flag [10:15] but, it will work [10:15] you may understand my dillema :-) [10:28] do I open MoarVM bugs on RT or GH? [10:29] both are fine i think [10:29] RT is used a little more? but I use github personally [10:29] stmuk: what's the bug? [10:29] the build has recently broken on FreeBSD [10:30] I have a crude fix [10:30] but not sure putting an #ifndef __FreeBSD__ in is that good nice [10:32] ah its reported with a similar fix anyway [10:32] https://github.com/MoarVM/MoarVM/issues/311 [10:36] oh, yes, i can recall that [10:39] i don't like the suggested fix; it'll end up with a long list of platforms that have fileno defined as a macro [10:39] iirc, jnthn recently added the native-fileno method on file handles [10:40] i'd prefer renaming the function fileno to native_fileno or something like that [10:46] *** virtualsue joined [10:46] *** Peter_R joined [10:57] It ended up as native-descriptor [10:58] I think we'll go with something like that as the name in Moar too [11:04] descriptor, that's the right term [11:12] yeah [11:12] brrt: if you have design decisions like that, why not "plan to throw one away"? ;P [11:12] see where it leads you? [11:12] hmmm [11:13] good enough point.... [11:13] i don't really see the big trouble with using a pseudotile for the inter-tile compiler work [11:14] it reminds me a tiny bit of how PHI nodes are handled in spesh [11:14] those are fake, too, in that they have an opcode of MVM_SSA_PHI, but there's actually multiple info structs allocated for them, because every PHI node can have a different number of arguments [11:14] well, there's another option, and i would like it best if it didn't involve a whole bunch of allocation work [11:15] which is, copy the arguments into an array..... [11:15] oh [11:15] that is not even a really bad idea [11:15] now that i think of it [11:16] i can just use a fixed-size array internal to the 'linear tile node' and pass it to the tile when it is time to emit it [11:16] that... fixes all my objections, which were (among other things) 'gee, isn't it a pain to manage a billion pointers for such a simple thing' [11:17] timotimo++ # you've just made my day :-D [11:17] <3 [11:18] (urgh, i'm entering the relative-overcaffeniation/under-caloric state, which means it is time for lunch) [11:25] i just had a tiny noms break my own self [11:25] i'm amazed i could pump out such good advice while nomming on some toast [11:29] * jnthn is still on breakfast... :) [11:30] i've only had a miniature breakfast so far :) [11:43] MoarVM: 1a896ff | jnthn++ | src/io/io. (2 files): [11:43] MoarVM: Fix build bustage where fileno is a macro. [11:43] MoarVM: [11:43] MoarVM: Turns out that C macros don't C so well. [11:43] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/1a896ff083 [11:44] jnthn: "still"? surely any meal is correct when one is in UGT? [11:45] True :) [11:48] with regard to https://github.com/MoarVM/MoarVM/issues/311 the parans fix of cygx might be a good temporary solution allowing *BSD until fileno is renaming (which appears to be in multiple places?) [11:48] s/ing/ed [11:50] stmuk: I already did it above [11:50] d'uh [11:50] :) [11:50] :) [11:50] Don't have a BSD to test it on though, so that'd be appreciated by anyone who does [11:51] oh I thought it was renaming the opcode as well [11:51] I can test [11:51] opcode should be harmless [12:06] * brrt has a bsd vm stashed somewhere [12:10] 1cc1: error: unrecognized command line option "-Wno-logical-op-parentheses" [12:10] ops [12:10] cc1: error: unrecognized command line option "-Wno-logical-op-parentheses" [12:11] OK, how do I get round that one... [12:12] --compiler=gcc [12:12] Upgrade to a newer clang? :P [12:12] OK, bigger F-up [12:12] no, really, what we need is someone who understands Win32 and *nix to produce (or "buy in") a better configuration system that doesn't pre-code a bunch of assumptions [12:12] but, failing that [12:12] I get to fix the gnu-make assumptions [12:12] oh wait [12:12] gmake is installed [12:12] it works on NetBSD I'm just waiting for the old clunky clang to finish [12:13] I'll ignore that bug for now too [12:13] clang36+ on FreeBSD is much faster than the default one [12:13] I'm not root [12:14] nwc10: which platform is this? [12:15] 7.2-RELEASE FreeBSD [12:15] and I'm not going to say *where*, as IIRC that's out of security [12:16] hahaha [12:16] OK [12:16] sorry, [12:16] build FAIL [12:16] there are gmake assumptions in the top level Makefile [12:16] it works (or worked) on FreeBSD 8.x [12:16] and some sort of "doesn't like gmake" assumptions somewhere lower [12:17] oh, odd [12:18] `gmake -` *works* [12:18] MoarVM builds [12:21] nwc10: if you wanted a modern toolchain you could probably use pkgsrc with a prefix of $HOME/pkg or whatever [12:22] I didn't know one could do that [12:23] its useful for people stranded on obsolete systems in industry :/ [12:24] for now, I seem to be getting a viable build with the default ancient(ish) toolchain [12:32] an old gcc is probably faster than an old clang anyway :) [12:40] nwc10, current version is 10.2 ? :-o [12:40] and yes, there are gmake assumptions [12:40] something like that [12:40] there is also special nmake magic i think [12:41] i wonder if the state-of-the-art in perl5 Configure.pl scripts has improved [12:41] i would *hate* to lose make, fwiw [12:41] also, i would equally hate to lose perl5-as-a-guaranteed-property-of-the-build process [12:42] I'd love to see the assumption of perl *5* gone from the build process. [12:42] but but but [12:42] I'm biased. [12:42] i use perl5 for my preprocessors [12:42] i'd have to rewrite everything in minilua :-P [12:42] "rebuild" is different from bootstrap [12:42] hmmm [12:43] I think removing perl 5 dependency from panda type installs should be first [12:43] lets just say i wouldn't like to checkin the generated headers from preprocessing templates / tile descriptions [12:43] I don't think needing perl 5 to build is unreasonable [12:44] thing is, something like luajit gets by *just fine* with a single makefile, no preconfiguring necessary [12:44] I *do* [12:44] configuring is imho an autotools legacy, and if we *can* get away without it, that would be better [12:45] but that's because I don't want to mandate that someone keeps having to maintain Perl 5, just so that Perl 6 can build [12:45] but this is at the 10 year timescale. [12:45] but you're right that Panda is the thing to start with [12:45] work from that end backwards, and see where things go [12:48] anyway moarvm builds on all 3 major BSDs (I deleted my DragonFlyBSD sadly) [12:50] stmuk++ [12:50] doing better than me [12:51] virtualbox++ [13:02] i think people will be maintaining perl5 for such timescales whether we want them to or not [13:02] oh yes, I don't disagree with *that* [13:03] but it shouldn't be forever [13:03] but start with Panda and work back [13:04] it should be viable for everything from NQP onwards to build without Perl 5, given that NQP already self-bootstraps with code written in NQP [13:04] and we're a long way from getting to that [13:04] which is "easy" (for a dubious value of easy) [13:04] "we" also need to address the toolchain code duplication between NQP and Rakudo [13:05] and all of that is probably more important and easier than any (semi) ideological purge of Perl 5 [13:05] I rather liked the current perl configure scripts since they were very clear and better than most perl 5 I've seen [13:05] they are pretty clear, but there are too many copies of the same code in 2 or 3 repositories [13:05] and there's too much assupmtion/not enough probing [13:06] (I'm biased) [13:06] but, as you said, start with Panda [13:06] certainly nicer than the usual auto* and the increasingly popular cmake [13:08] I think leont has a prove6 type thing already [13:17] * brrt nods [13:17] yeah, the whole thing can be made much more robust [13:17] but there is so much encoded knowledge there.... [13:18] pretty much nobody knows all that about all platforms [13:39] * jnthn back [13:40] *** geekosaur joined [13:41] *** geekosaur joined [13:54] ohai jnthn [13:54] o/ brrt [13:57] MoarVM: 8aa4cd4 | jnthn++ | / (3 files): [13:57] MoarVM: Configure bits for \n -> \r\n on output. [13:57] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/8aa4cd48e3 [14:23] MoarVM: e7ebeb9 | jnthn++ | src/ (12 files): [14:23] MoarVM: Implement input newline translation. [14:23] MoarVM: [14:23] MoarVM: Applies to all I/O handles except sockets. [14:23] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e7ebeb957a [14:36] *** travis-ci joined [14:36] MoarVM build failed. jnthn 'Implement input newline translation. [14:36] https://travis-ci.org/MoarVM/MoarVM/builds/97224178 https://github.com/MoarVM/MoarVM/compare/8aa4cd48e324...e7ebeb957a29 [14:36] *** travis-ci left [14:36] I bet that'll be heisenbuild... :P [15:01] MoarVM: 6935e94 | jnthn++ | / (30 files): [15:01] MoarVM: Translate newline \n -> \r\n on output. [15:01] MoarVM: [15:01] MoarVM: When doing I/O for anything except sockets, and only when configured [15:01] MoarVM: to do so (meaning "on Windows builds"). [15:01] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6935e941df [15:05] heh, not heisenbuild :) [15:07] MoarVM: 49f10b9 | jnthn++ | src/strings/normalize.c: [15:07] MoarVM: Fix missing intialization. [15:07] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/49f10b9b86 [15:16] *** travis-ci joined [15:16] MoarVM build failed. jnthn 'Translate newline \n -> \r\n on output. [15:16] https://travis-ci.org/MoarVM/MoarVM/builds/97232730 https://github.com/MoarVM/MoarVM/compare/e7ebeb957a29...6935e941dfc1 [15:16] *** travis-ci left [15:16] jnthn: t/moar/03-line-seps.t fails [15:16] not ok 14 - \r\n separator used [15:17] Yeah, that test is now wrong [15:17] Fixed locally [15:27] *** travis-ci joined [15:27] MoarVM build failed. jnthn 'Fix missing intialization.' [15:27] https://travis-ci.org/MoarVM/MoarVM/builds/97233863 https://github.com/MoarVM/MoarVM/compare/6935e941dfc1...49f10b9b8651 [15:27] *** travis-ci left [16:20] *** domidumont joined [16:21] *** zakharyas joined [16:22] commute & [16:25] *** zakharyas joined [16:53] *** zakharyas joined [16:59] Seems we we aren't especially validating typevars properly [16:59] In the bytecode validator [16:59] Meaning we get away with spitting out bytecode for various things with very dubious behaviors [17:00] Like num32 lex being read into a num64 register, and vice versa [17:00] Without the appropriate coercion [17:01] that could be kinda bad [17:02] Yes, it's why num32 mis-behaves in https://rt.perl.org/Ticket/Display.html?id=124084 [17:03] oh, it seems it does happen for reg reg, but not for lex reg or reg lex [17:10] I wonder how much chaos fixing this will cause [17:19] Well, nqp's make test still passes [17:19] And it catches the issue [17:19] C:\consulting\rakudo>perl6-m -e "my num32 $n; say $n;" [17:19] Bytecode validation error at offset 18, instruction 4: [17:19] operand type 40 does not match register type 48 [17:21] ideally we'd complain at the compiler stage rather than at the bytecode validation stage [17:21] That's a nice to have :) [17:22] Running bytecode that's full of it is just really wrong :) [17:22] right! [17:22] Feel free to patch mast/compiler.c or so [17:22] (I'll probably only do that now if it makes debugging easier) [17:24] Well, NQP builds and passes tests with the patch. Now to see if Rakudo builds. [17:25] Phew, yes, it does :) [17:26] Blows up make test in a couple of places [17:33] *** zakharyas joined [17:34] And a couple of spectests, it seems [17:34] But seems everything it catches is genuine bad code-gen [17:35] very good [17:35] I'd guess the nativecall fails that it results in match up with native call fails on BE [17:40] *** virtualsue joined [17:44] MoarVM/sized-native-fixes: ee0afc2 | jnthn++ | src/core/validation.c: [17:45] MoarVM/sized-native-fixes: Validate lexical types properly. [17:45] MoarVM/sized-native-fixes: [17:45] MoarVM/sized-native-fixes: This means that we catch invalid code-gen for storing wrongly sized [17:45] MoarVM/sized-native-fixes: values into lexicals, and vice versa. [17:45] MoarVM/sized-native-fixes: review: https://github.com/MoarVM/MoarVM/commit/ee0afc28ab [17:45] MoarVM/sized-native-fixes: d23cbe2 | jnthn++ | src/core/interp.c: [17:45] MoarVM/sized-native-fixes: Implement truncate/extend ops for num32. [17:45] MoarVM/sized-native-fixes: review: https://github.com/MoarVM/MoarVM/commit/d23cbe23b8 [17:45] *aha*, them [17:45] .oO( The sun never sets on Big Endian ) [17:46] Enough for now...will look at the code-gen fixes we need later [17:48] (MVMnum64)GET_REG(cur_op, 2).n64; # do you mean MVMnum32 here? [17:48] yes [17:49] MoarVM/sized-native-fixes: d6544f5 | jnthn++ | src/core/interp.c: [17:49] MoarVM/sized-native-fixes: Fix cast; JimmyZ++. [17:49] MoarVM/sized-native-fixes: review: https://github.com/MoarVM/MoarVM/commit/d6544f54d9 [17:49] That's exactly why I'll worry about the rest later :P [17:50] ;) [17:50] dinner & [18:35] jnthn: dare I suggest the nativecall fails are also related to what you saw on windows with the deserialization cache thing XD XD XD XD *Christmas Cheer*** [18:35] (yes I'm still gonna try to repro on windows) [18:52] *** leont joined [18:55] *** zakharyas joined [19:04] *** zakharyas joined [19:05] *** vendethiel joined [19:19] *** zakharyas1 joined [19:35] *** virtualsue joined [19:37] diakopter: Sadly, I highly doubt it...the bustage I've uncovered will be cross-platform. [19:49] *** FROGGS joined [19:52] oh [20:00] *** virtualsue joined [20:01] jnthn: ASAN likes (OK, tolerates with the usual disdain) your branch so far [20:02] :) [20:25] *** virtualsue joined [20:54] *** virtualsue joined [20:58] *** zakharyas joined [21:25] wow, just running nqp's Configure on an ubsan-enabled moar spews tons of errors [21:25] it didn't do that a few weeks ago, iirc [21:26] laods of misaligned loads [21:26] src/core/validation.c:156:12: runtime error: load of misaligned address 0x7fcb21c463d9 for type 'MVMuint16', which requires 2 byte alignment [21:27] that's opcode = *(MVMuint16 *)val->cur_op; [21:28] and callsites_equal() passing null pointers to memcmp [21:29] https://github.com/joyent/libuv/issues/414 [21:29] that explains the UDP issues [21:30] jnthn: could that explain the blowup? https://gist.github.com/FROGGS/bc20e5799a39d97b94f2#file-gistfile1-txt-L16-L23 [21:30] went to bed [21:33] ahh :o) [21:37] *** colomon joined [21:38] ubsan output from nqp Configure.pl run: http://paste.scsys.co.uk/502894 [21:39] wow [21:39] gcc 5.2.1-23 on debian testing [21:39] these sanitizers are so cool [21:40] MoarVM: 016bb5e | FROGGS++ | src/core/bytecode.c: [21:40] MoarVM: init cd->arg_flags so callsites_equal wont complain [21:40] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/016bb5ec9c [21:40] sadly, this does not help [21:41] well, no, it's still passing null pointers to memcmp [21:41] if arg_flags == NULL is a valid state, callsite_equals needs to guard against that [21:46] ilmari: nopaste the error list? [21:46] diakopter: 21:38 < ilmari> ubsan output from nqp Configure.pl run: http://paste.scsys.co.uk/502894 [21:47] ahah [21:48] gah [21:52] FROGGS: there are other places where arrays that would be zero-sized are not allocated, then add_foo functions blindly memcpy them [21:55] *** travis-ci joined [21:55] MoarVM build passed. Tobias Leich 'init cd->arg_flags so callsites_equal wont complain' [21:55] https://travis-ci.org/MoarVM/MoarVM/builds/97320646 https://github.com/MoarVM/MoarVM/compare/49f10b9b8651...016bb5ec9cae [21:55] *** travis-ci left [21:56] although in other places it does malloc(0) [22:27] but malloc(0) is allowed to return NULL anyway, so we can't rely on that [22:28] *** colomon joined [22:54] FROGGS: https://github.com/MoarVM/MoarVM/pull/312 [22:55] FROGGS: fixes all null pointer memc(py|mp) in callsite.c and spesh/ [23:13] MoarVM: 31bdb0f | (Dagfinn Ilmari Mannsåker)++ | src/spesh/ (3 files): [23:13] MoarVM: Avoid memcpy(x, NULL, 0) in spesh [23:13] MoarVM: [23:13] MoarVM: Although most compilers sensibly do nothing in this case, it is [23:13] MoarVM: undefined behaviour according to the standard, and UBSAN complains. [23:13] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/31bdb0f7b4 [23:13] MoarVM: c571752 | (Dagfinn Ilmari Mannsåker)++ | src/core/callsite.c: [23:13] MoarVM: Don't try to compare empty flag sets [23:13] MoarVM: [23:13] MoarVM: This avoids calling memcmp() with NULL pointers, which is undefined [23:13] MoarVM: behaviour, even is the count is zero. [23:13] MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c5717527a2 [23:13] ilmari: I merged it [23:14] show me a couple of the 6model ones you're not sure about [23:26] ilmari: which 6model things [23:32] diakopter: the ones in src/6model/serialization.c:concatenate_outputs [23:33] diakopter: ubsan only flagged the one for repos_table [23:33] this was just during 'make' of nqp [23:34] maybe ubscan divined that it couldn't definitively determine that the size sent to memcpy would always be zero when the pointer was invalid [23:35] I don't think ubsan does that kind of analysis [23:35] oh [23:35] null pointers to memc(py|mp) are undefined even when size == 0 [23:35] well, is it a problem to send invalid pointers to memcpy when the size is 0? oh [23:35] I see [23:35] yes, it's explicitly undefined, presumably for speed/ease of implementation [23:38] yeah, the check for zero would need to be *somewhere*; might as well make the caller do it [23:39] and often you know that it can't be zero, so don't need to check [23:40] but yes, should add those checks to the 6model serialization too [23:41] most of those memcpy usages could theoretically be zero [23:51] 'nqp make' is up over 200k lines of ubsan output [23:51] mostly misaligned access [23:51] well hopefully it decreases non-linearly XD [23:52] I'll upload the full output pluss deduplicated by source code line