github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:00 reportable6 left, reportable6 joined 00:10 pamplemousse joined 00:15 pamplemousse left 00:55 Kaiepi left 00:56 Kaiepi joined 05:31 robertle left 06:00 reportable6 left 06:02 reportable6 joined 06:28 brrt joined
brrt \o 06:53
07:01 sena_kun joined
nwc10 o/ 07:17
07:44 zakharyas joined 07:52 sena_kun left
brrt ohai nwc10 08:37
timotimo nwc ohai10 08:38
brrt timohaimo 08:39
nine There's the possibility that the threaded GC issue is not actually a real issue but just some overly sensitive debug checks 08:49
nwc10 I (have lost newor again) 08:50
IIRC the asserts were put in by me to confirm my understading/code analysis 08:51
the thing IIRC was that there were 2 pointers in every object header
1 that was only needed *during* GC (the forwarder)
and 1 only needed outside of GC (forget which, commits should be obvious)
timotimo there may be a #define that brings the not-unionized format back
nwc10 and I figured that the two could be the same in a untion 08:52
and everythings gets smaller by one pointer
beware though - the MoarVM code is almomst too clever for its own good
it sort of just about gets alignment right on x86_64, x86 and ARM
particularly w.r.t. doubles in C structures bodies, which *may* be straight after the header 08:53
or might be in a malloc'd block pointed to be the fowaring pointer thingy
and chaning the header size (at one point in 2014) broke all of this
I *think* that the asserts aren't needed 08:54
but I do not know enough about the "other thread actually moving the item" 08:55
you'd need to ask jnthn
it might be the "stealing work" thing that I vaguely remember
(network bounced at least once in all of that)
09:40 brrt left, brrt joined 09:50 brrt left 10:55 AlexDaniel is now known as test{}, test{} is now known as AlexDaniel
nine nwc10: ok, thanks! That fits with my understanding which I've gathered over the course of the past week. 10:55
11:38 zakharyas left 12:00 reportable6 left 12:03 reportable6 joined 12:13 brrt joined 12:49 pamplemousse joined 13:10 zakharyas joined
Guest37021 nine: are we out of GC related errors now? 13:20
nine Guest37021: at least out of those that I can make reliably reproducable 13:39
There are a few spec test files that fail with high-level errors which may be GC related or may just be triggered by MoarVM's slowness when the GC runs all the time 13:40
13:51 robertle joined
Guest37021 cool, I suspect that we should be able to close a few MoarVM issues thanks to your fixes 13:59
nine Oh, that would be nice indeed :) 14:00
Guest37021 perhaps M#1062 14:01
synopsebot M#1062 [open]: github.com/MoarVM/MoarVM/issues/1062 malloc error in t/04-nativecall/06-struct.t
Guest37021 I guess I'll try to repro them and if I can't I'll have to figure out which of your commits fixed the issue 14:03
nine I think that one's still unsolved. It's about inlined structs, isn't it? 14:05
or stuff inlined into structs
Guest37021 yeah, I was too optimistic, that one is still open 14:06
nine: how about M#1140, might that have been fixed 14:13
synopsebot M#1140 [open]: github.com/MoarVM/MoarVM/issues/1140 Panics and segmentation faults when working with sockets and threads
14:25 lucasb joined 14:26 brrt left 14:30 brrt joined 15:21 robertle left 15:29 pamplemousse left 16:39 brrt left 16:59 robertle joined 17:31 chloekek joined 18:00 reportable6 left 18:02 reportable6 joined 18:07 zakharyas left 18:39 brrt joined
Kaiepi i fucked up the ipv6 pullreq at some point after unit testing it, i'll force push the fix once make spectest's done 18:56
19:19 sena_kun joined
nine Guest37021: not, not either :( But that's actually pretty much the same error as another one we talked about: spesh still being active in a finished (and free()'d) thread. The one where timotimo++ suggested adding checks to the few places where spesh allows for GC 19:26
timotimo there's three places where spesh allows for GC, they look like "GC_SYNC_POINT(TC)" 19:27
19:27 lucasb left 19:35 zakharyas joined
nine timotimo: but...that's either not entirely true, or there are bugs. I know of at least one backtrace where the GC was triggered by deserialization in spesh 19:37
timotimo that's not very good 19:39
we used to MVM_panic in that case
or it was in the code i wrote to go back out of speshing a frame to deserialize and then retry
19:47 pamplemousse joined 19:51 pamplemousse_ joined 19:54 pamplemousse left 19:56 pamplemousse_ is now known as pamplemousse
nine timotimo: ah, we do ineed MVM_panic in that case: gist.github.com/niner/61258e039e02...891b260f52 20:03
But from a user's perspective, that's not exactly better :/ 20:04
timotimo: wouldn't this be an appropriate fix? gist.github.com/niner/ef6dd2466fcf...82a0fde041 20:08
A WVal that's used in speshed code should by definition be somewhat long lived. Otherwise the code wouldn't get speshed in the first place 20:09
Kaiepi ok the ipv6 pullreq now passes make spectest for real now 20:12
note to self: don't make last minute style fixes after unit testing and expect it to always still work 20:14
nine Kaiepi: I guess we've all been there :) 20:15
20:20 zakharyas left
timotimo nine: the code gets speshed even if the wval is in a branch that's never taken, which is very often the case with exception throwing branches 20:26
in theory it'd be fine to just not set the facts when the object isn't deserialized yet 20:27
since it'd be only not deserialized yet if it's in a cold branch inside of a hot routine
sena_kun o/ 20:28
is anyone interested in a severe memory leak which is likely to be a regression?
nine timotimo: oh, very good insights!
timotimo i could have had that insight years ago … 20:29
nine timotimo: so what :)
sena_kun or any clues on how to get more helpful info are very appreciated, tools 20:30
nine sena_kun: well, would you be able to bisect it? 20:31
sena_kun nine, what do I need for it? I don't have whateverable installed if that's needed. 20:32
nine just git and a lot of time
sena_kun nine, it'll kill the process for me, right? as it just hangs and starts to eat memory, not something to terminate. :S if yes, I am ready for executing commands 20:33
timotimo sena_kun: the moarvm heapanalyzer could perhaps help
nine sena_kun: no, git bisect is pretty simple. It helps you find a faulty commit by being smart about trying them. You start with a known faulty and a known working one and it helps you narrow it down. But the actual test is a manual thing. 20:35
sena_kun nine, ah, I see
nine sena_kun: To be precise, the actual test can be automated, too, but that's not required
sena_kun let me look at moarheapanalyzer...
nine timotimo: if you meant this fix, then it does indeed work! gist.github.com/niner/ce8fc4f422ae...399606aa0f 20:36
timotimo first gotta either --profile=heap which will spit out heap snapshots every time GC runs, or "use Telemetry; snap :heap" whenever you're interested in a snap of the heap
nine: that could do it
nine Will run some tests and then commit
sena_kun timotimo, does process need to terminate? I am running it with `--profile=heap`, but after some time and ctrl-c there is no snapshot file or I'm blind. 20:37
> (the filename will be at the end of the program output). 20:38
ah, likely it needs to. :S
timotimo with --profile=heap it needs to terminate for the snapshot to be readable (something my in-progress work fixes) 20:39
sena_kun oh... that pretty much means git bisect then... 20:41
timotimo not necessarily 20:45
you don't have to have the process terminate when using snap :heap 20:46
sena_kun timotimo, I'll check it a bit later, thanks! 20:50
timotimo OK 20:51
Geth MoarVM: 654a136d13 | (Stefan Seifert)++ | src/spesh/facts.c
Fix panic in spesh when deserialization causes GC during fact finding

Deserialization may trigger garbage collection or at least allow for it to happen, but in spesh there are very few places where GC is actually allowed. If a WVal is not yet deserialized at the time the code its used in is speshed then we may assume that it's used by a cold branch inside of a hot routine. In that case its facts are probably not as important, so we may as well get by without them. Forgoing those facts gets rid of the GC issue.
Many thanks to timotimo++ for the valuable insights and the solution!
nine That's my 24th GC related fix in a row :D 20:52
20:55 pamplemousse left, pamplemousse joined 21:01 pamplemousse left 21:09 Altai-man_ joined 21:11 sena_kun left 21:36 brrt left
japhb nine: I think you mean your 24th act of awesomeness in a row. ;-) 21:46
timotimo on the 24th day of christmas ...
21:56 MasterDuke joined
MasterDuke wow! just installed my new hardware. so much faster! 21:56
i ran a rakudo spectest right before powering off with the old hardware, it took 600s. ran another after rebooting with the new hardware, it took 133s 21:57
timotimo, sena_kun: --profile=<something> now creates a regular profile with the given filename. you need --profile-kind=heap to get a heap snapshot 22:15
dogbert17 MasterDuke: congrats 22:19
22:20 pamplemousse joined
dogbert17 what's your parse time when building Rakudo? 22:20
MasterDuke 76s before, 36s after 22:21
dogbert17 gee, that's fast 22:22
MasterDuke yeah, pretty happy with how much faster it actually is 22:24
dogbert17 I can imagine :-)
22:32 BinGOs_ joined 22:37 BinGOs_ is now known as BinGOs, BinGOs left, BinGOs joined 22:52 pamplemousse left 22:55 Altai-man_ is now known as sena_kun 23:00 pamplemousse joined 23:08 sena_kun left, sena_kun joined 23:16 chloekek left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!