|
00:44
colomon joined
01:08
[Coke] joined
01:34
colomon joined
02:30
MadcapJake joined
06:51
domidumont joined
07:18
zakharyas joined
|
|||
| timotimo | jnthn: are we interested in .moarvm files that segfault or hang moar when it tries to load it? | 09:48 | |
| jnthn | timotimo: Yeah, that shouldn't really happen | 09:49 | |
| We go to quite some effort not to read past the ends of buffers when picking apart bytecode files | |||
| timotimo | well, i'm running the American Fuzzy Lop to corrupt some moarvm files :) | ||
| jnthn | Any hangs/SEGVs so far? :) | 09:50 | |
| timotimo | 330 unique crashes, 218 unique hangs | ||
| but i think i might be doing it wrong | |||
| i've fed it the nqp stage0 as a starting point | 09:51 | ||
| jnthn | hah | ||
| timotimo | but there's interdependencies | ||
| oh, neat | 09:52 | ||
| timo@hack:~/fuzzing/fuzzing-input$ ~/fuzzing/install/bin/moar ~/fuzzing/fuzz-state/crashes/id:000329,sig:11,src:000006,op:flip1,pos:64117 | |||
| Segmentation fault | |||
| at the moment it seems to mostly be finding new hangs, which means it's only able to do like 30 tests per second | 09:53 | ||
| it used to do 10x that in the midle, and 20x that at the beginning | |||
| i have the pretty graphs :) | 09:54 | ||
| t.h8.lv/afl/ | 09:55 | ||
| there they are | |||
| if i read the status screen correctly, it's still processing the 6th input file | 09:57 | ||
| which means we're looking at easily a year of fuzzing time :P | |||
| i'm considering re-running the fuzzing but giving it --dump on the commandline | 09:58 | ||
| that'd concentrate the fuzzing effort on actually loading one bytecode file and if i guess correctly it'd skip trying to load dependencies | 09:59 | ||
| ah. yes, it would | |||
| arnsholt | When I discussed this with jnthn a while back, it seemed to me generating really small bytecode files would probably be useful as well | 10:10 | |
| timotimo | yeah | ||
| arnsholt | Sadly I never got around to actually implementing that | ||
| timotimo | i chopped the dumper down to size so it only forces deserialization of most frames and actualization of some strings | 10:14 | |
| with the fuzzing run that doesn't just --dump, i expect many crashes would be located around the place where it starts to run bytecode | 10:16 | ||
| i wonder if i should keep that run going at all? | |||
| hack.p6c.org/~timo/afl1/ - running a full moar *.moarvm | 10:50 | ||
| hack.p6c.org/~timo/afl2/ - running moar --dump *.moarvm, but with most of the bytecodedump code cut out | |||
|
11:46
mtj__ joined
12:37
zakharyas joined
|
|||
| diakopter | timotimo: that's really neat | 13:13 | |
| timotimo: feel free to email me some of those crashes/hangs | 13:14 | ||
| timotimo | diakopter: hack.p6c.org/~timo/crashes_hangs.tar.gz - those are the ones we have so far for --dump | 13:53 | |
| diakopter | timotimo: did you try afl-tmin | 14:19 | |
|
15:04
domidumont joined
|
|||
| timotimo | no | 15:27 | |
| did you try out any of those crashing cases yet? | 15:37 | ||
|
15:39
synopsebot6 joined
|
|||
| diakopter | m: EVAL q[] while 1 | 16:23 | |
| camelia | rakudo-moar 61d231: OUTPUTĀ«Memory allocation failed; could not allocate 385792 bytesā¤Ā» | ||
| diakopter | timotimo: well, no, I was hoping you could run afl-tmin to try to minimize the cases | 16:24 | |
| timotimo | well, i can try | 16:25 | |
| i have to run it for each individual result file? | 16:26 | ||
| diakopter | no idea | ||
| I just saw it mentioned in the readme in the .tar.gz | |||
| timotimo | it seems like that stuff works fine | 16:28 | |
| File size reduced by : 1.78% (to 11276 bytes) | |||
| Characters simplified : 99.45% | |||
| the first file is now littered with "0" ascii bytes :) | |||
| diakopter | lol | ||
| timotimo | hack.p6c.org/~timo/simple_crash.moarvm | 16:29 | |
| take a look yourself | |||
| diakopter | r: hack.p6c.org/~timo/simple_crash.moarvm | ||
| camelia | rakudo-jvm 5eaa15: OUTPUTĀ«cannot connect to eval server: Connection refusedā¤Ā» | ||
| ..rakudo-moar 61d231: OUTPUTĀ«===SORRY!=== Error while compiling /tmp/tmpfileā¤Confusedā¤at /tmp/tmpfile:1ā¤------> http:ā//hack.p6c.org/~timo/simple_crash.moarvmā¤ expecting any of:ā¤ colon pairā¤Ā» | |||
| timotimo | haha, as if. | ||
| diakopter | :D | ||
| timotimo | Program received signal SIGSEGV, Segmentation fault. | 16:30 | |
| MVM_bytecode_unpack (tc=tc@entry=0x6037c0, cu=0x661cb0) at src/core/bytecode.c:878 | |||
| 878 MVM_ASSIGN_REF(tc, &(cu->common.header), cu_body->hll_name, | |||
| i'll be AFK for a bit. | |||
| fwiw, the "--dump only" hasn't discovered a new path in 1 hour and 18 minutes | 16:31 | ||
| but it's only at the 5th input file, which it has finished 90% of apparently | |||
| so we'll see what happens once it reaches the 6th | |||
| diakopter | how many input files are there | ||
| timotimo | 10 or 9 | ||
| diakopter | billion? | 16:32 | |
| timotimo | files | ||
| diakopter | billion exabytes? | ||
| timotimo | that's the initial files, i just copied over the stage0 from nqp | ||
| diakopter | hehehe | ||
| timotimo | hack.p6c.org/~timo/ - the "total paths"/"pending paths" will tell you how many files it's got in its queue | ||
| i suspect the "full run" has stumbled upon the interpreter and is now joyously exploring our instruction set | 16:33 | ||
| diakopter | lolz | 16:47 | |
|
17:00
domidumont joined
|
|||
| timotimo | damn it. | 17:06 | |
| it's now still in the 5th file, but in a different strategy | |||
| "arith 8/8" | |||
| diakopter | hehe | 17:07 | |
| timotimo | it'll take 24M execs and it does about 600 execs per second | ||
| diakopter | ETOOMANY | ||
| timotimo | it's already reached 2% !! | 17:08 | |
| it seems like it found a new path a quarter hour ago | |||
| so that's nice | |||
| but it'll still take forever until it noms up the remaining ~600 paths | 17:09 | ||
| and all in all it's at 0.77% - that number goes down every time a new path gets discovered, though | 17:10 | ||
| i should probably have used afl-cmin first to reduce the incoming files a bit, so it doesn't have to mess with all of the files | 17:11 | ||
| [+] Narrowed down to 9 files, saved in 'minimized_corpus'. | 17:12 | ||
| gee, thank you! | |||
| diakopter | ur a corpse | 17:13 | |
| timotimo | you're too kind | 17:14 | |
| diakopter | XD | ||
| timotimo | i'm not patient enough for fuzzing :| | 17:17 | |
| it's really not something you should sit in front of and attentively watch, that's for sure | 17:18 | ||
|
17:48
leont joined
18:16
FROGGS joined
|
|||
| timotimo | i'm now minimizing all those crashes i've found so far for the "only --dump" case | 18:39 | |
| diakopter | :D | 18:55 | |
| timotimo | the files fall into two categories so far: one where the space saving is like 2%, the other where the space saving is like 25% | 19:02 | |
| *** Error in `../../install-changed-dump/bin/moar': malloc(): memory corruption: 0x00007f3c41b60610 *** | 19:08 | ||
| we have a .moarvm file that gives us that | |||
| at least i think so? | |||
| diakopter | splendid | ||
|
19:21
zakharyas joined
|
|||
| timotimo | i'm quite worried that the speed of the second run might take a gigantic dip like the first run did when it reaches file number 6 | 19:25 | |
|
19:33
TimToady joined
|
|||
| timotimo | "auto extras" | 20:05 | |
| i have no idea what that is or does | |||
|
20:29
domidumont joined
|
|||
| timotimo | diakopter: want the minimized crash files? | 20:39 | |
| diakopter: hack.p6c.org/~timo/minimized_dumpon...hes.tar.gz | 20:41 | ||
| diakopter | timotimo: cool, thanks :) | 21:12 | |
| dalek | arVM: a5cf746 | timotimo++ | src/core/bytecode.c: bytecode_unpack: bail out if HLL name string index is invalid |
22:17 | |
| jnthn | timotimo: Nice catch :) | 22:19 | |
| timotimo | the first 10 crashes afl found were because of this | ||
| actually | 22:20 | ||
| all the crashes it found were this ... | |||
| all 16 crashes | |||
| but there's 20 unique crashes by now. i can bring the tarball up to date soon | 22:22 | ||
| oh, interesting, a segfault in reand_int32 called by finish_frame | 22:25 | ||
| of course with our lazy deserialization, exceptions about corrupt bytecode files will now fly at random moments in time :) | |||
| fun!! | |||
| jnthn | :) | 22:27 | |
| timotimo | missed a few ensure_can_reads that i now littered across the function | 22:29 | |
| hum. where do i reach the rs i need to pass to ensure_can_read when i'm in finish_deserialize? | 22:32 | ||
| i thought the StaticFrame or maybe the CompUnit has a pointer to the ReaderState, but it doesn't seem to be the case | |||
| hm, maybe we just store the full size in some place | 22:33 | ||
| is data_size the right number here? | 22:34 | ||
| no, serialized_size is the right one i bet | |||
| hm. when we stumble upon that problem, we can't "cleanup_all" like we usually do | 22:40 | ||
| cleaning up the whole compunit is probably not so easy when we're potentially in the middle of code in there? | |||
| now to figure out the reentrant mutex auto-unlocking stuff | 22:55 | ||
| oh, i can just have it as an argument. for this single-purpose helper function it should be fine | 22:56 | ||
| Unhandled exception: Could not finish deserializing frame: Read past end | 23:00 | ||
| [Inferior 1 (process 1511) exited with code 01] | |||
| dalek | arVM: 9c2368b | timotimo++ | src/core/bytecode.c: spread some range checks in MVM_bytecode_finish_frame |
23:06 | |
| arVM: bab36ec | timotimo++ | src/core/bytecode.c: finish_frame: don't try to set flags beyond num_lexicals |
|||
| timotimo | well, damn. now i went ahead and b0rked it for "real" code :( | ||
| Unhandled exception: Could not finish deserializing frame: Read -1292547922 past end | 23:09 | ||
| holy moly. | |||
| this is a bit less obvious than i had hoped. i'll revert and branchify | 23:11 | ||
| dalek | arVM: e42d949 | timotimo++ | src/core/bytecode.c: Revert "spread some range checks in MVM_bytecode_finish_frame" This reverts commit 9c2368bd53bcbb17ed4a53af9c55b2ad07265ce0. That commit got the limit calculation wrong and made legit files unreadable. |
23:16 | |
|
23:17
travis-ci joined
|
|||
| travis-ci | MoarVM build failed. Timo Paulssen 'finish_frame: don't try to set flags beyond num_lexicals' | 23:17 | |
| travis-ci.org/MoarVM/MoarVM/builds/121568422 github.com/MoarVM/MoarVM/compare/a...b36ec54124 | |||
|
23:17
travis-ci left
|
|||
| timotimo | travis is yet to report it, but my revert fixed the build again | 23:31 | |