|
00:31
reportable6 joined
01:26
FROGGS joined
01:57
ilbot3 joined
|
|||
| Zoffix | timotimo: "t/spec/S07-hyperrace/stress.t does this usually take ages" No it usually takes like 3s. It's a regression bug; something with GC: github.com/rakudo/rakudo/issues/1709 | 02:20 | |
|
03:21
bloatable6 joined,
coverable6 joined,
nativecallable6 joined,
quotable6 joined,
notable6 joined,
unicodable6 joined,
committable6 joined,
evalable6 joined,
releasable6 joined,
benchable6 joined,
reportable6 joined,
undersightable6 joined,
greppable6 joined,
squashable6 joined,
bisectable6 joined,
statisfiable6 joined
|
|||
| samcv | i succeded in getting debian MIPS running in qemu | 04:26 | |
| samcv now has so many VM's. solaris, openbsd, freebsd, dragonflybsd and now mips debian. | |||
| Zoffix | :) | 04:30 | |
| samcv | installing debian took at least 1.5 though :) | 04:33 | |
| *hours | |||
| possibly more | 04:34 | ||
| you have to boot directly from the kernel since there's not real mips hardware implementation. well bios/bootloader implemetation. | |||
| so you have to copy the kernel and initrd out of the install disk to install it. and then after installed have to mount the VM hd and copy out its kernel and initrd to boot from that | 04:35 | ||
| working on implementing SipHash. thinking about stripping out the code which makes sure it returns the same values no matter the endianess. since it really shouldn't matter unless we're testing | 04:36 | ||
| also: -bash: shutdown: command not found :( | 04:38 | ||
| yay after 48 minutes compiling it fails :) | 05:59 | ||
| ./libmoar.so: undefined reference to `dcCallbackThunkEntry' | |||
| FROGGS | perhaps use libffi | ||
| samcv | FROGGS: well it should include dyncall right? | 06:00 | |
| FROGGS | it does, but not all function are implemented for all architectures and operationg systems | ||
| see the chart here: www.dyncall.org/ | 06:02 | ||
| dyncallback is no supported everywhere | |||
| samcv | FROGGS: it says it should be supported | 06:03 | |
| big endian mips at least which is what i'm using | |||
| FROGGS | but your compiler says the opposite :o) | ||
| I also had a mips machine and s390x and others, and I often had to switch to libffi | 06:04 | ||
| samcv | yeah it compiles fine but i guess that function isn't getting defined | ||
| FROGGS | aye | 06:05 | |
| samcv | FROGGS: that did it. MVM installed | 07:05 | |
| FROGGS | \o/ | ||
|
07:24
shareable6 joined,
releasable6 joined
08:28
greppable6 joined,
undersightable6 joined
09:13
dogbert2_ joined
09:14
evalable6 joined
09:18
reportable6 joined
10:00
reportable6 joined
10:42
domidumont joined
10:48
domidumont joined
11:16
notable6 joined
12:43
evalable6 joined
14:10
evalable6 joined
14:47
zakharyas joined
15:32
bartolin joined
16:11
releasable6 joined,
undersightable6 joined
17:54
zakharyas joined
18:44
brrt joined
|
|||
| brrt | good * | 18:44 | |
| i have ehm, some questions about MVMStaticFrame static env | 18:47 | ||
| jnthn probably knows something about that, though is a tad tired from travel :) | 18:48 | ||
| brrt | i'm wondering if i can formulate my questions faster than i can figure it out by git grep | 18:49 | |
| what i'm doing is, i'm following through Tux's CSV test SEGV, which is an inlined frame (DELETE-KEY) that is referencing a static env lexical (flag=1) that is null | 18:50 | ||
| timotimo also appeared | 18:51 | ||
| brrt | ohai timotimo | ||
| jnthn | Hmmm...that's...interesting. What path is it taking to look at the static env? | 18:52 | |
| brrt | it is calling MVM_frame_vivify_lexical, from an inlined frame | 18:53 | |
| nwc10 | good *, #moarvm | ||
| brrt | the inliner is postcircumfix{ } | ||
| \o nwc10 | |||
| i have before-and-after spesh logs if you want to see | 18:54 | ||
| samcv | i'm almost done porting siphash to MoarVM | ||
| jnthn | Hmm...I thought that path had code to check "are we in an inline" and find the correct static lexical env | ||
| samcv | had to experiment around with some different implementations, find one sufficiently fast and in c, and then had to make it work with moarvm properly. i tried a streaming capable implementation but it was 4x slower than our previous function, probably more. it added a huge amount of time. but i adapted some other one to let us add 64 bits at a time | 18:55 | |
| and then we close it out with 7 to 0 bytes and get the hash | 18:56 | ||
| brrt | it does actually, yes | ||
| actually, it assumes we are in the correct inline | 18:57 | ||
| jnthn | Hm, so I wonder if that part is doing the right thing | ||
| brrt | i'm decently positive that it is | ||
| or well | |||
| hmmm | |||
| nwc10 hopes that jnthn is suitably holidayed. Or whatever the verb is | 18:58 | ||
| jnthn | nwc10: Yes, in that it was a nice time by I was ready to come home by the end of it. :-) | 19:00 | |
| s:1st/by/but/ | |||
| brrt: If it is, then I'm wondering why we'd not trip over said nullness the first time around | 19:01 | ||
| brrt | me too | ||
| jnthn | (as in, pre-opt) | ||
| brrt | hmmm | 19:02 | |
| gist.github.com/bdw/e3c1b02617169d...b70fa944f5 | 19:07 | ||
| this is what i have on it now | |||
| samcv | jnthn: if we want something inlined i should probably put it in src/ right | 19:12 | |
| and not in 3rdparty. and i've modified it to add new functions to the 3rdparty code, so it might make sense to make it MVM_STATIC_INLINE and just include it in src? | 19:13 | ||
| jnthn | samcv: I think that what'd matter more is how it's compiled in rather than where it is | 19:18 | |
| samcv: If you're talking about the hash code, though, by now what we have is so far removed from the original uthash sources that it probably would be sensible to stick it under stc/ | |||
| samcv: That way we keep 3rdparty as "things you might expect as a packager to replace with a package dependency", which the hash stuff is totally not | 19:19 | ||
| samcv | but i added github.com/samcv/csiphash/blob/leu....c#L51-L67 | ||
| so we have three commands to work on the hash. to initialize the variables, to add 64 bits at a time and finish it out | 19:20 | ||
| jnthn | Ah, you're looking at changing the hash function we use? | ||
| samcv | ah ok | ||
| yeah | |||
| which is part 2 of the PR i submitted for randomizing our hash secret | |||
| which should give us full coverage to prevent DOS attacks | 19:21 | ||
| jnthn | OK; quick summary of what ciphash offers over what we do now? | ||
| (Also, thanks for taking this on.) | 19:22 | ||
| samcv | it's not easily reversible by the same method that can cause a hash attack specified in github.com/MoarVM/MoarVM/pull/841 (i got into some basics on how it works) | ||
| the attack that is | |||
| jnthn | Aha, OK | ||
| From the PR: | 19:23 | ||
| samcv | so siphash is secure as long as they don't know the secret. while other hash functions like we use could be exploited remotely. randomizing makes it one step harder. but then a more secure hash function is step 2 to remove the ability to exploit | ||
| jnthn | "This change does | ||
| increase the ease of this attack greatly and makes the hash secret | |||
| unpredictable." | |||
| Did you mean decrease? :) | |||
| samcv | oops | ||
| jnthn | Phew :P | ||
| samcv | MVM_HASH_EXPLOIT=1 | 19:24 | |
| MVM_HASH_EXPLOIT_EASE=easy/hard | |||
| haha | |||
| jnthn | :D | ||
| I'd better review carefully ;) | |||
| I'll look at it properly tomorrow when I'll be able to give it the concentration it deserves. | 19:25 | ||
| samcv | ok :) | ||
| nwc10 | what does the spectest make of it? | ||
| samcv | is that a question for me? | ||
| nwc10 | (sorry, this might not seem like the most intelligent or useful question, but experience of Perl 5 was that making hashes actually behave like the documentation (didn't) guarantee exposed a lot of code that was making unwarranted assumptions) | 19:26 | |
| for anyone here | |||
| samcv | ah | ||
| i've found no issues | |||
| brrt | nwc10: it still does.... | 19:27 | |
| samcv | at least wiki says Perl has siphash as a configurable on compile | ||
| not sure if that's the reason it's set that way nwc10 ? | |||
| nwc10 | I've forgotten all the recent variants of configuration options | ||
| jnthn | I think having to work over MoarVM/JVM/JS will have shaken out a lot of our spectests, thanks to the backends using different algorithms. | 19:28 | |
| brrt | i not so long ago gave a perl5 crash course wherein participants *insisted* that a 'map' was an ordered key-value collection | ||
| turns out c++ std::map does it that way | |||
| nwc10 | but what I remember was that if you start enabling randomisation of ordering, you can expose a lot of bugs | ||
| jnthn: aha yes. an advantage I'd not considered | |||
| jnthn | But yeah, the ecosystem can indeed potentially have fallout from this, as most real use of Perl 6 is currently MoarVM-based. | 19:29 | |
| samcv | but i think the hash keys are themselves not specific to the impl. i mean we use 32 bit keys atm. and i'll prolly switch to 64 bit since that's what siphash uses. but atm my branch that has randomizer + siphash is currently on uint32_t size hashes. i just assign the 64 bit uint to a 32bit uint | ||
| nwc10 | to be clear (if I wasn't) I think that this is the right direction to be taking, but it might have some short term pain | ||
| jnthn | I guess the toaster will tell us :) | ||
| samcv | i don't know any way that we expose our hash keys? | 19:30 | |
| jnthn | releasable6: when is the next release, kind bot? | ||
| releasable6 | jnthn, I cannot recognize this command. See wiki for some examples: github.com/perl6/whateverable/wiki/Releasable | ||
| brrt | we should probably embrace that pain | ||
| nwc10 | and sorry, I fail to have looked at the branch | ||
| samcv | i mean if we did that would be a super bad and make exploit much easier :) | ||
| jnthn | releasable6: next | ||
| releasable6 | jnthn, Next release in ā5 days and ā23 hours. 7 blockers. 224 out of 236 commits logged | ||
| jnthn, Details: gist.github.com/293f200fa3ce841cae...ed47c0e901 | |||
| brrt | samcv: currently our hash seed is constant, which means that the unspecified hash order is effectively deterministic across runs | ||
| samcv | ah | ||
| jnthn | Hmmm....we probably want to merge this *after* that release :) | ||
| nwc10 | samcv: Perl 5 doesn't expose the hash keys. But various folks demonstrated "interesting" ways of recovering them from minimal amounts of data | ||
| samcv | ok thank you. now i understand | ||
| brrt | iirc there is some perl5 hackery that you can use to get the hash of a string | 19:31 | |
| samcv | so when we get a list of hash keys it is always the same order every run? | ||
| brrt | currently, probably, yes | ||
| samcv | or does rakudo hide that? | ||
| brrt | m: my %h = (foo => "Bar", quix => "quam"); say %h.keys; | ||
| camelia | (quix foo) | ||
| samcv | m: my %h = (foo => "Bar", quix => "quam"); say %h.keys; | 19:32 | |
| camelia | (quix foo) | ||
| brrt | m: my %h = (foo => "Bar", quix => "quam"); say %h.keys; | ||
| camelia | (quix foo) | ||
| nwc10 | also, I should add (hello Internet and logs) that I am (for some reason) still on the perl 5 security list, so I've seen some stuff that isn't public | ||
| that wasn't phrased well | |||
| specific exploits that we didn't make public at the time we hardened the implementatin to not be exposed to them | 19:33 | ||
| but some folks are still running 5.10.1 or earlier, and you can't fix that for some of the more esoteric problems | 19:34 | ||
| samcv | m: my %hash; my $a = 'a'; for ^20 { %hash{$a .= succ} = $a; }; say %hash.keys | ||
| camelia | (p k h g s q e d m f c o n u t b j i r l) | ||
| samcv | yes every start of rakudo it has a different order | ||
| with the randomization patch | |||
| though also. if synthetics are added in a different order those would change as well | 19:35 | ||
| but hopefully no modules rely on the order being constant | |||
| nwc10 | None should. Likely at least one wrongly does :-/ | 19:37 | |
| (and needs its bug exposed) | |||
| samcv | i'm also making a #define so we could make the hashes the same even on big endian arch with the change. though by default i modified this siphash impl so instead of always flipping the bytes on big endian it only does if we set a define. and if the define is set then we'll flip the 32 bit graphemes as well | ||
| nwc10 | I don't think that Parrot ever *did* this, but something Dan said that he wanted to do, was to explicitly code to randomise any ordering that was not documented as being ordered (or stable, or watever) | ||
| so that nothing came to accidentaly rely on something that was not guaranteed | 19:38 | ||
| samcv | option probably not to be used much since it's really not needed. but if someone wanted hashes to always be the same, or for debugging it could maybe be useful | ||
|
19:38
Kaiepi joined
|
|||
| samcv | anybody know somewhere i could get access to a big endian machine remotely? | 20:45 | |
|
20:51
Kaiepi joined
|
|||
| samcv | hmm casting a 64bit integer to a 32 bit could cause endianess issues yes? | 21:03 | |
| jnthn | via a straight cast, from MVMint64 to MVMint32? | 21:07 | |
| samcv | MVMuint64 to MVMuint32 | ||
| jnthn | I'd be surprised | ||
| I mean, it can of course truncate the upper 32 bits, but I think what's consider upper should be consistent. Would have to check the C standard to see what, if anything, is promised, though. | 21:08 | ||
|
21:09
MasterDuke joined
|
|||
| jnthn | If doing hashing, I'd far more likely expect endian issues if somehwere a sequence of 8 bytes is treated as a 64-bit integer pointer | 21:09 | |
| I can tiredly glance some code if it'd help :) | 21:10 | ||
| samcv | so this test file works fine on big endian github.com/samcv/csiphash/blob/leu....c#L37-L53 | 21:11 | |
| i swap the bytes for the 32 bit integers that are part of the test and ensure it gives the same hash | |||
| but for some reason it's not working in MoarVM. or doesn't seem to be. though i'm compiling MoarVM there again which will take a while. and will put in some code that will test a known string directly | |||
| and this is it used in moarvm github.com/samcv/MoarVM/blob/highw...2891-L2959 | 21:12 | ||
| for the time being i'm using a char[16] as the static key just to make sure there's no issues there | 21:13 | ||
| MVM_MAYBE_TO_LITTLE_ENDIAN_32 basically sets the endianess to little endian. and it works in that test.c file | 21:14 | ||
| so that makes me think the issue must be in the moarvm code since test.c works fine | 21:15 | ||
| hmm maybe i should try using a loop in the test code and calling siphashadd64bits instead of the function that grabs everything in the test file. | 21:17 | ||
| jnthn: ok got a failure :) | 21:26 | ||
| this code github.com/samcv/csiphash/blob/leu....c#L40-L61 needs to function like this small code here github.com/samcv/csiphash/blob/leu...c#L97-L106 | 21:28 | ||
| maybe it's the casting going on? though it's just casting a pointer | 21:29 | ||
|
21:30
Zoffix left
|
|||
| samcv | ok that was my mistake. so it actually does work fine. so i guess i'll wait for MoarVM to compile again | 21:32 | |
| OK i think i figured out the issue. finally. | 21:35 | ||
|
22:26
lizmat joined
|
|||
| samcv | \o/ awesome. ok now that headache is over | 22:37 | |
| timotimo | ah, the performance regression that's still there after i fixed the wvalfrom thing is just because the new ops aren't jitted yet %) | 23:08 | |
| well, maybe not "just", but it's pretty likely | |||
| OK, i've got a template for it, that's fun | 23:29 | ||
| didn't want to put it into the lego jit because it needs to grab a spesh slot and we don't have something simple for that yet | |||
| Geth | MoarVM: 1525500479 | (Timo Paulssen)++ | src/jit/core_templates.expr add an exprjit for sp_getwvalfrom |
23:30 | |
| MasterDuke | cool. how's the performance now? | 23:31 | |
| timotimo | looks like back to normal | ||
| MasterDuke | nice | 23:32 | |
| timotimo | sadly, getstringfrom also needs an implementation now :) | 23:35 | |
| also, rakudo's make test seems unhappy? | 23:36 | ||
| maybe only with my local changes | 23:38 | ||
| yup | 23:39 | ||
| hum :| | 23:41 | ||
|
23:44
travis-ci joined
|
|||
| travis-ci | MoarVM build failed. Timo Paulssen 'add an exprjit for sp_getwvalfrom' | 23:44 | |
| travis-ci.org/MoarVM/MoarVM/builds/366935164 github.com/MoarVM/MoarVM/compare/f...25500479fd | |||
|
23:44
travis-ci left
|
|||
| timotimo | hum. not good | 23:46 | |
| Geth | MoarVM: 94637c24be | (Timo Paulssen)++ | src/core/interp.c fix wrong int size for string register in getstringfrom |
||
| timotimo | it's not working well at all, wow | 23:54 | |
|
23:55
Kaiepi joined
|
|||