github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
01:33 Kaeipi left, Kaeipi joined 01:47 leont left 01:50 Kaeipi left, Kaiepi joined 05:35 sena_kun joined 05:41 Altai-man joined 05:44 sena_kun left
MasterDuke hm. i've been experimenting with setting custom memory functions for gmp 08:15
gist.github.com/MasterDuke17/68ca8...23d7680713 is my current diff 08:16
they do get called (i added some fprintfs and breaks on them in gdb to confirm) 08:17
however, `say 2⁴⁵⁵³⁵³⁵³⁴⁵³⁶⁴⁵³⁵³⁴⁵` dies (backtraced added to above gist) 08:18
i haven't been able to trace through all the macros yet. i thought maybe it was because it was trying to alloca, but the documentation says that the alloca codepath uses the regular (or custom if set) memory management functions if the value is too big 08:21
i also rebuilt it with `--enable-alloca=debug`, which is supposed to use the regular mem functions in all cases, no change 08:22
anyone have an idea why the memory functions i added aren't being called when this dies? oh, and i confirmed that they are in fact being called before the pow is evaluated 08:26
nwc10 I have read this but I have no idea. I'll have to repeat 08:29
repeat [Coke]'s excellent observation "I am at least not you, but that's all I have here." in relpy to "anyone who is not me and feels competant"
and I have limited time sadly need to get other stuff done 08:30
MasterDuke no worries 08:38
hm. might it be a problem that the memory functions are beng set twice? 08:41
should `mp_set_memory_functions(&MVM_gmp_malloc, &MVM_gmp_realloc, &MVM_gmp_free);` only happen once, but `gmp_tc = tc;` happen for each thread? 08:44
nwc10 that seems like a good guess, but that's all I can say 08:46
MasterDuke well, no change in behavior...
too bad, hoped that was it 08:48
nwc10 me too - it sounded plausible
MasterDuke what's interesting is that if i set a breakpoint on my MVM_gmp_realloc, the backtrace shows the immediately preceding frame is in __gmpz_realloc, realloc.c:67 08:53
oh. that line is after the error checking that aborts! 08:55
so even setting custom mem functions doesn't bypass the abort? that sucks 08:57
ok peeps, what do we want to do now? seems like we could patch gmp, but then we can't use `--has-gmp` (or just say that you may get aborts instead of exceptions if doing so?)? come up with some heuristics about values that could potentially cause the aborts and then catch them in moarvm (e.g., not allowing an exponent over X for nqp::pow_I, even if 09:05
it might be ok for some bases)?
afk for a bit
09:12 synopsebot joined, synopsebot left 09:16 brrt joined 09:42 sena_kun joined 09:43 Altai-man left 10:16 brrt left 11:12 patrickb joined 11:39 leont joined
timotimo m: say 4553535345364533345 / 1024 / 1024 / 1024 / 1024 11:59
camelia 4141416.27094480404139
timotimo do i understand correctly that that pow would get us a number that's a few thousand petabyte big?
perhaps we can replicate the error check that gmp does and put it before our call to gmp_pow 12:00
and say "sorry folks, if you have a machine with many petabytes, you won't be able to use it all for big integer math" 12:01
12:07 patrickb left 12:09 patrickb joined
timotimo on intel chips, x86_64 physical addresses are limited to 52 bits, and virtual addresses to 48 bit, so trying to support something with an exponent that high would be futile anyway 12:10
and in order to do sensible math on such big numbers we'll need two times to four times more memory anyway, for intermediate results and such 12:11
12:13 patrickb left 12:53 vesper11 left 13:41 Altai-man joined 13:43 sena_kun left 15:03 brrt joined 15:48 brrt left 16:51 zakharyas joined 17:42 sena_kun joined 17:44 Altai-man left
MasterDuke m: say 71148989771320869 / 1024 / 1024 / 1024 / 1024 20:14
camelia 64709.62923351259542
MasterDuke 71148989771320869 is the new_alloc parameter to realloc 20:15
but yeah, still big
but the error check gmp does is just `new_alloc > INT_MAX`. it isn't in gmp_pow, it's in realloc 20:18
so to replicate it exactly, we'd have to figure out what values of base and exponent lead to such a large allocation. and then that might be different for gmp_mul, etc 20:19
nine Sounds tedious and error prone... also may still be totally worth it 20:47
I guess catching the SIGABRT wouldn't work? 20:50
MasterDuke according to SO (stackoverflow.com/questions/335662...ion-in-c), abort() can't be caught. the signal can, but the function detects that and tries again/overrides 20:59
(though i haven't tried yet, guess i should...)
oh, maybe it can work if we longjmp back, not return from our handler 21:01
moon-child timotimo: newer intel has 57-bit virt space, so it should be ... possible 21:09
(for some definition of possible. You probably want a specialised library that prioritizes memory use over speed, though) 21:11
21:15 zakharyas left, sena_kun left 21:20 sena_kun joined
MasterDuke it also depends on the mathematical operation you're doing. e.g., there's some algorithm for expmod that can do some sort of running calculation, it doesn't require calculating the full exponentiation first. so we might be able to expmod with a larger exponent than we can pow 21:25
timotimo right 21:29
21:40 sena_kun left
MasterDuke mostly works gist.github.com/MasterDuke17/734c8...5f495d70c7 21:44
does cause a new warning though: src/math/bigintops.c: In function ‘MVM_bigint_pow’:src/math/bigintops.c:121:16: warning: variable ‘i’ might be clobbered by ‘longjmp’ or ‘vfork’ [-Wclobbered] 121 | mpz_t *i = tc->temp_bigints[idx]; 21:45
timotimo cursed 21:50
21:50 patrickb joined
timotimo is signal handling not process wide? 21:51
lizmat should be, no?
timotimo that would be bad :) 21:52
lizmat not process wide you mean? 21:55
21:56 patrickb left
timotimo well, if one thread is doing integer math with gmp and another aborts, for example from a malloc error or whatever, then the handler for int math would run 21:57
MasterDuke does it help to make that jmp_buf also __thread? 22:00
also, is there a way to suppress those `gmp: overflow in mpz type` messages? catch stderr while the signal handler is active? 22:04
timotimo i imagine the handler would run then, and just find the jmp_buf to be nulled out?
i think futzing with stdout and such would be an even worse idea :) :)
MasterDuke i guess if you're going to catch the exception in raku-land what's a little stderr output to deal with also 22:17
hm. what happens if you `longjmp(NULL)`? 22:18
is this information useful? "If, and only if, the savesigs argument provided to sigsetjmp() is nonzero, the process's current signal mask is saved in env and will be restored if a siglongjmp() is later performed with this env." 22:19
well, i'm to bed. if anyone has suggestions for how to make this robust, let me know and i'll probably be able to do some experimenting in the next couple days 22:40
timotimo without actual cooperation from gmp we will also have a hard time doing cleanup for whatever gmp may have done before triggering the abort 23:18
since it is written so that the program just shuts down when it aborts 23:19