github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:05 anatofuz left 00:06 anatofuz joined 00:11 anatofuz left 00:18 anatofuz joined 00:22 anatofuz left 00:23 anatofuz joined 00:25 anatofuz left, anatofuz joined 00:48 lucasb left 00:49 anatofuz left
Geth_ MoarVM/more_efficient_serialization: 0b47ff9ec3 | (Timo Paulssen)++ | src/6model/serialization.c
serialization: smaller storage for empty arrays/hashes
 01:09
timotimo that's the 0.075% win or whatever that was 01:11
01:13 anatofuz joined 01:34 tellable6 left, anatofuz left 01:38 tellable6 joined 01:54 anatofuz joined 01:57 anatofuz left 02:11 anatofuz joined 02:14 anatofuz left 02:19 anatofuz joined 02:21 anatofuz left 02:37 anatofuz joined 02:41 anatofuz left 02:43 anatofuz joined 03:00 anatofuz left 03:14 anatofuz joined 03:18 anatofuz left 03:20 anatofuz joined 03:31 anatofuz left 03:32 anatofuz joined 03:37 anatofuz left 03:51 anatofuz joined 03:55 anatofuz left 03:57 anatofuz joined 04:03 anatofuz left 04:12 anatofuz joined 04:17 anatofuz left 04:41 anatofuz joined 04:45 anatofuz left 04:54 anatofuz joined 04:59 anatofuz left 05:08 anatofuz joined 05:12 anatofuz left 05:22 anatofuz joined 05:27 anatofuz left 05:30 anatofuz joined 05:37 sena_kun joined 05:41 anatofuz left 05:47 anatofuz joined 06:03 squashable6 left 06:06 squashable6 joined 06:11 anatofuz left 06:14 domidumont joined 06:15 anatofuz joined, domidumont left 06:16 domidumont joined 06:23 domidumont left 06:29 anatofuz left 06:40 domidumont joined, anatofuz joined 06:41 anatofuz left, anatofuz joined 06:43 anatofuz left 06:45 anatofuz joined, anatofuz left 07:11 anatofuz joined 07:12 anatofuz left 07:28 anatofuz joined 07:33 anatofuz left 07:39 anatofuz joined 07:44 anatofuz left 08:08 sena_kun left 08:11 zakharyas joined 08:52 anatofuz joined 08:55 sena_kun joined 09:05 anatofuz left, anatofuz joined 09:06 anatofuz left 09:07 anatofuz joined 09:27 MasterDuke left 09:47 anatofuz left, anatofuz joined 09:49 anatofuz left, anatofuz joined 09:52 anatofuz left 10:01 anatofuz joined 10:08 domidumont left
nine Ok, I'm starting to become desparate here. To recap: I have an easily reproducible case of a mixin's STable pointing at a corrupt HOW. The STable in question seems to be freed by the GC but reappears later in an invokee. I can only reproduce the issue if multiple threads create similar mixins. But even completely deactivating the mixin cache has no effect. 10:17
lizmat so, a race condition on creating identical subtypes ? 10:20
perhaps deactivating the mixin cache exacerbates the problem ? 10:21
nine Apparently. But I can't find anything that a race could be on. The mixin cache was the obvious candidate for a shared data structure that gets modified in the process, but it doesn't look like it's at fault
Deactivating the mixin cache seems to have no effect whatsoever, except for the expected slow down of course 10:22
lizmat hmmm... if it slows down, but then does not make the problem go away or be worse, then maybe it isn't a race condition after all ? 10:24
maybe the problem is then *accessing* the same subtype from different threads?
jnthn I wonder if it's anything to do with the parameterization handling at the VM level 10:25
nine jnthn: what would that be? 10:28
10:32 Guest69469 joined
jnthn nine: The use of setparameterizer in MixinCacheHOW 10:34
Though you mentioned you disabled that and still saw the issue. 10:35
So maybe not.
Did you have an isolated example?
Is this the segv you did on the evalbot yesterday?
nine that's the one, yes 10:36
Ok, to be clear, what I disabled was this code: github.com/MoarVM/MoarVM/blob/mast...tric.c#L63 10:37
I didn't know that there was a higher level cache, too
jnthn Ah, well, it mostly just wraps around the thing you mentioned 10:38
nine I have found quite a number of issues while looking including GC issues with find_method and friends and a deadlock but not what I was actually looking for 10:44
Odd... turning this github.com/rakudo/rakudo/blob/mast...ns.nqp#L41 into my $mixin_type := 0 && self.is_composed($obj) to never use the cache breaks the build: array[int] *must* be parameterized 10:45
So instead I create a new cache object for every call. That gets me far enough through the build to run my test case and it still fails 10:51
timotimo nine: does the problem also happen when valgrind is active? because there's a what-points-here command you can send to it via its gdbserver interface 11:30
if you know the address of the object that had been freed, you can check what has pointers at it 11:31
Guest69469 nine: I guess that the GC stacktraces were of no use then?
nine Guest69469: unfortunately not :/ 11:39
timotimo: oh, valgrind definitely reports them as Invalid reads 11:40
timotimo: how does the gdbserver interface thing work?
11:42 zakharyas left
timotimo valgrind --vgdb=yes --vgdb-error=0 prog 11:45
gdb prog
(gdb) target remote | vgdb
i don't think i've tried this yet tbh
valgrind.org/docs/manual/manual-cor...ver-simple 11:46
i recommend configuring moarvm with --valgrind if you haven't yet 11:47
cool, the valgrind gdbserver supports read and write watchpoints; any number of them, and with no length limit 11:55
12:03 domidumont joined 12:05 domidumont left 12:06 domidumont joined 12:32 lucasb joined
nine I do get a list of memory loations pointing at the STable in question. All but one of them are on different threads' stacks 12:49
timotimo on stacks, huh 12:53
that's not fun
have you been able to find which stack frames correspond to the locations?
nine in frame #5, created by MVM_interp_run (interp.c:104) 12:56
13:17 anatofuz left 13:27 zakharyas joined 13:36 domidumont left
nine Oh, this may be a deciding hint: the STable is destroyed by MVM_gc_gen2_destroy during a thread's destruction! 13:45
Yes, I totally fail to reproduce the issue using the thread pool. 13:50
timotimo oooh 13:55
i'm now using the uppermost 4 bits of the discriminator as "extra data" storage space 13:56
nine So it may either be some bug in thread cleanup. Or maybe still a missing write barrier as without the gen2roots entry, the pointer will not be updated.
timotimo on thread destruction we're actually trying to copy everything out to another thread
nine But....where are pointers updated? I see code for copying memory and for taking over the gen2roots, but not for actually updating the roots
else { /* note: we don't have tests that exercise this path yet. */ 13:57
13:57 domidumont joined
nine Or do those objects actually stay where they are and the memory areas are taken over by the receiving thread? 13:58
timotimo not sure 14:00
i would assume one way to do it - if not the way we do it - is to set a forwarder on all of the objects and then run a GC
but simply merging the gen2 pages to another gen2 allocator could also work 14:01
gen2 are per thread, right?
nine I guess so
Anyway, have to leave for a couple hours
timotimo good hunting though 14:02
14:03 zakharyas left
timotimo m: say 14630232 - 14594536; say 54592 - 54440; say 120792 - 120584 14:28
camelia 35696
152
208
timotimo savings for core.c, core.d, and core.e settings
m: sub infix:<->($a, $b) { $a * 100 / $b }; say 14630232 - 14594536; say 54592 - 54440; say 120792 - 120584 14:29
camelia 100.24458469
100.279206
100.172494
timotimo m: sub infix:<->($b, $a) { $a * 100 / $b }; say 14630232 - 14594536; say 54592 - 54440; say 120792 - 120584
camelia 99.75601207
99.721571
99.827803
timotimo m: sub infix:<->($b, $a) { 100 - $a * 100 / $b }; say 14630232 - 14594536; say 54592 - 54440; say 120792 - 120584
oops
oh well 14:30
14:34 camelia left 14:36 camelia joined
camelia (timeout) 14:37
timotimo that took a while 14:42
Guest69469 m: is-prime(-1) 14:57
camelia WARNINGS for <tmp>:
Useless use of "is-prime(-1)" in expression "is-prime(-1)" in sink context (line 1)
Invalid number of rounds (100), valid range is 0..256

in block <unit> at <tmp> line 1
15:08 domidumont left 15:19 domidumont joined 15:35 MasterDuke joined 15:38 domidumont left 15:57 Kaiepi left 15:58 Kaiepi joined 16:11 domidumont joined 16:26 domidumont left 16:28 domidumont joined 16:37 domidumont left 16:53 robertle joined
timotimo urgh, the serialization doesn't work out right any more with my changes :| 16:57
17:43 nebuchad` joined 17:44 nebuchadnezzar left, nebuchad` is now known as nebuchadnezzar
Geth_ MoarVM/more_efficient_serialization: 20a340be9a | (Timo Paulssen)++ | 2 files
store 4 extra data bits in the discriminator

currently used for small integers (-1 through 14), the size of object/int/str arrays, and hashes (only the ones directly implemented in the serializer code)		 18:12
MoarVM/more_efficient_serialization: 3aa0169786 | (Timo Paulssen)++ | src/6model/serialization.c
fix an error message (that should never happen)
MoarVM/more_efficient_serialization: 93900418c9 | (Timo Paulssen)++ | 3 files
attempt to refactor discrim extra data; broken
timotimo i'm not entirely sure if the heap dump is correct here, the code refs array in an MVMCompUnitpoints at some BOOTCode that points, via being something's Caller, to data that's been created during the current run 20:00
20:10 domidumont joined 20:11 domidumont left 20:30 sena_kun left
timotimo gist.github.com/timo/d587372f66562...ecffe4f391 - does this look reasonable? 20:43
20:45 Ven`` joined
timotimo Permanent Roots (Permanent Roots:0) (Frame) (374912) 20:51
okay *this* definitely doesn't look right
21:09 MasterDuke left
samcv when is it estimated for MoarVM release? 21:31
21:33 anatofuz joined
AlexDaniel samcv: oh I don't know… there are a bit too many blockers right now 22:10
22:16 anatofuz left 22:17 Ven`` left 22:30 anatofuz joined 22:34 anatofuz left 22:56 anatofuz joined 23:20 Kaiepi left 23:23 Kaiepi joined 23:41 anatofuz left 23:51 anatofuz joined 23:52 lucasb left 23:55 anatofuz left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!