github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:13 leont left 04:10 frost-lab joined 04:34 jdv79 joined 07:09 kanopis joined
nwc10 good *, #moarvm 07:40
MasterDuke ah ha. turn on a bunch of debug defines and now i get a `Invalid owner in item added to GC worklist`. these are so much fun to debug... 08:09
nwc10 debugging is hard, let's go, oh, erk, lockdown. Debugging it is then. 08:10
rr?
MasterDuke yes, lockdown 3.0. going to be a fun time for all 08:11
i'll probably end up in rr, but i haven't added all that much code since it was working fine. maybe i'll be able to see what needs fixing directly
08:31 domidumont joined 08:33 brrt joined 08:50 zakharyas joined 09:19 Altai-man joined 09:29 kanopis left
nine Huh....it seems like objects can end up in the deserialization workloop multiple times. This can cause memory leaks as the deserialize REPR functions usually assume that they are called on uninitialized objects (repossession, which is a cause for multiple deserialize calls, calls gc_free before deserialize) 11:13
brrt \o nine 11:14
11:15 ilogger2 joined
jnthn nine: I think the only case that should ever happen is repo 11:25
(Which as you note does clean up first)
Could try disabling lazy deserialization to see if it goes away; if yes, that at least narrows it down 11:38
MasterDuke jnthn: i suspect my current panics are due to a missing MVM_ASSIGN_REF for dealing with the new pointer that i now use to access the arg guards and candidates. i'm afraid i still don't completely understand MVM_ASSIGN_REF, but you likely know if/where/how a new one (or more) is needed 11:40
something like ASSIGNing the candidate list and the arg guard to the new struct? 11:41
nine jnthn: taking it from the other direction, what mechanism is there to actually prevent this? E.g. we serialize an object that references a closure which references the same object. 11:45
jnthn MasterDuke: It's needed for each new object that is being referenced 11:47
MasterDuke: I don't think the number of MVM_ASSIGN_REF needs to change, nor the second argument (which collectable owns it), just the final argument (where the reference can be found) 11:48
nine: My recollection (it's been a while since I looked at this) is that we have a stub step that happens immediately upon first mention, and that only happens once 11:49
(per object)
So if it's stubbed we know it's already scheduled for deserialization or some such
In non-lazy mode we just unconditionally stub every object
And so they all end up on the worklist
Lazy is stub on first reference and add to worklist at that point 11:50
nine Yes, the stub step is run only once for the object. Otherwise we'd end up with 2 distinct objects after all. Then we deserialize and use MVM_sc_get_object for getting referenced objects. Ordinarily this will get deserialized objects from the root objects array and only run deserialization if necessary. But if we're already deserializing this check is skipped and we always deserialize. 11:51
github.com/MoarVM/MoarVM/blob/mast.../sc.c#L216 11:52
11:58 sena_kun joined
MasterDuke jnthn: maybe it doesn't need to change (it was added when MVMSpeshCandidate was made a REPR). `MVM_ASSIGN_REF(tc, &(spesh->common.header), new_candidate_list[spesh->body.num_spesh_candidates], candidate); spesh->body.spesh_candidates = new_candidate_list;` the ASSIGN was added. now that second line is instead 11:59
`new_cands_and_arg_guards->spesh_candidates = new_candidate_list; spesh->body.spesh_cands_and_arg_guards = new_cands_and_arg_guards;`
jnthn MasterDuke: No, probably no need there 12:03
nine Ah, of course, we cannot rely on the object we find in the roots array, as that may still be a stub 12:16
But then I wonder what actually keeps us from running into an endless loop there. 12:17
jnthn Probably because we don't always demand objects? 12:20
nine Simply checking the worklist for presence of the index before blindly adding it seems to get around this issue 12:50
13:59 zakharyas joined
nine A simple rakudo-m -e 'signal(SIGINT).tap: &exit' will not be able to close the event loop because of the active singal handle, causing asan to complain. I wonder which layer is the appropriate place for forcing that handle to close. 14:01
14:02 brrt joined
nwc10 good *, brrt 14:03
I read that Ruby 3.0 is out. But is it still using the JIT that is "fork gcc, generate a .so, and the dlopen() that .so back into the process?" - I couldn't find out if they've improved on that.
brrt good * nwc10 - I think it was 14:06
I'm not sure that's going to make a lot of people happy to run it in production
nwc10 from what I read, and this was a while back, and might be out of date, this JIT did not actually pass the "but does it run Rails?" test 14:10
(well, "does it run Rails faster?" in this case)
because in the end, this approach runs out of address space to map in the shared objects. 14:11
jnthn nwc10: afaict, yes, that's still the approach 14:15
nine: The event loop has a list of all active handles, I believe
nine: I forget exactly what's in there, but in theory iterating through that and closing things would do it 14:16
Would need to arrange for it to happen on the event loop thread
nine Hm...yes, it does. That's how uv_loop_close notices that there are open handles and refuses closing 14:18
jnthn nine: Ah, I wasn't clear. I mean that MoarVM also has its own active handles list 14:19
nine Oh....I guess that would be tc->instance->event_loop_active 14:20
jnthn yeah, sounds about right 14:21
nwc10 jnthn: ah OK. That seems about as sound a strategy for a JIT as Perl 5.6 was for Unicode. 14:22
brrt (what was the problem with perl 5.6 and unicode?) 14:24
nwc10 probably ticks the box on "can run a mandlebrot much faster" which seems to be the use case for several JITs and code generators which add static typing to a dynamic language and declare this useful to be "faster"
jnthn I've not read up on what the larger strategy is. For example if it's to batch things
I'm not sure how you get from where they are to "in-process"
nwc10 brrt: 1) no IO, 2) no Unicode in has keys 3) /./ and everything else in regexes often actually acted upon the underltying internal represetation (ie UTF-8) 14:25
and a load of other bugs
jnthn Oh, maybe load libclang or something and then get the machine code output in memory?
nwc10 jnthn: good point, me neither. The compiler-and-shared-object approach appears to be a dead end
I have no idea what libclang can do
but
jnthn I don't know if it can do *that*
brrt llvm has not yet ever worked well for JITs 14:26
jnthn But it wouldn't surprise me if it could
nwc10 I don't yet remember anyone writign a blog post to say "We used LLVM for our dynamic language JIT and it worked really well first time"
brrt lol
nwc10 Both Pyston (the first version) and Safari both found that they needed to add another layer before LLVM, didn't they?
jnthn Pretty sure I remember reading about a JavaScript JIT where they found this, at least. 14:27
nwc10 brrt: you said it more accurately. I was maybe a bit cynical
OK, it was a JS JIT, but we can't remmber which oen
jnthn They had to tier it because the llvm approach was good for high quality code
But not latency
And that matters in a browser context
nwc10 yes, thanks for typing tht for me, and better than me 14:28
jnthn But yeah, the whole "things using X are fast, so use X and you'll be fast too" is one of those annoying over-simplifications. 14:32
brrt :-) 14:33
MasterDuke wait, what?! gluing a giant wing onto the back of my automatic, diesel, french minivan won't make it go faster? /me goes off to cancel that order... 14:41
14:48 brrt left
jdv79 goes back into lurking via the public logger 14:50
14:53 lucasb joined
nine Some cases are easy to fix, but it's not clear if a fix is correct, or if there's something better we could do. E.g. Supply.interval(1).tap(-> { say "hi" }); sleep 3; will fail with an arity error and we'll leak tc->active_handlers. Now it's easy to implement cleanup of active handlers when we destroy a thread. But is it OK for them to still be there in the first place? 14:58
jnthn I don't understand the "when we destroy a thread" part 15:06
All handlers for async things are processed by the event loop thread
So destroying that is a global operation
nine This is about exception handlers 15:07
jnthn If you have a timer that you just want to have run until shutdown, then you'll leak it anyway
Even without the exception
Well, "leak", because if it's application lifetime you don't really care
But yes, it's easy to mess up and leak things when doing async if you call .tap directly, which is why I push react/supply so strongly. 15:08
In this particular case you'd get the exception handled, have the subscription cleaned up, then have it rethrown outside of the `react`
nine In this case we're leaking the MVMActiveHandler allocated here: github.com/MoarVM/MoarVM/blob/mast...ons.c#L362 15:09
jnthn oh, sigh, we talked about another thing called active some minutes ago 15:10
nine Yeah...and both things have handlers
jnthn Yeah, that's something else. I don't know what to do about that :)
nine The English language clearly has too few words :)
nwc10 rename it, to avoid the confusion! :-)
jnthn Are we in this situation because we call "exit"? 15:11
nwc10 (that was a reply to jnthn half in jest, but nine appears to have made it seem sensible)
jnthn I don't think active handlers should generally leak, but if the exception handler itself does an `exit` then we never unwind the stack
nine Ah, yes, something's doing an nqp::exit 15:12
Geth MoarVM/asan_fixes: 4a7b65bd2c | (Stefan Seifert)++ | src/core/threadcontext.c
Clean up active exception handlers when garbage collecting a thread
 15:14
nine So that's probably a sensible approach
MasterDuke btw, wouldn't it be good to move the malloc down to where `ah` is first used since MVM_frame_find_invokee can throw?
nine Indeed. Good thing, we're no longer bound by C89's shackles 15:18
MasterDuke oh, the MVM_frame_find_invokee call could also be moved down 15:26
nine But then we could end up leaking that MVMActiveHandler again? 15:27
MasterDuke yeah, that was just sort of talking/thinking out loud 15:33
[Coke] was there a recent change to which C standard we're adhering to? 15:55
[Coke] not a C developer
15:57 Altai-man joined
nine We switched to C99 a year ago 15:59
15:59 sena_kun left 16:13 brrt joined
[Coke] Danke. 16:48
brrt I never write enough C anymore :-( 17:17
nine brrt: you're young. Plenty of time left to write C :0 17:32
:)
cog Or rust, which is, at last, a clean C 17:36
moon-child rust is a bastard child of c++ and ocaml 17:37
zig somewhat closer to a clean c
brrt lol, I wonder if we can discuss rust without things turning it into a flamewar 17:38
moon-child :)
brrt I too think rust is much, much more like C++, in design and intended usage, than to C
moon-child (rust also has sprinklings of clean and cyclone)
brrt C is a fairly raw language at the end of the day 17:39
nine I fear the commit about active exception handlers is a bit too focused after all. There are plenty of other kinds of special return data that we can leak. 17:40
17:51 patrickb joined, patrickb left 18:12 brrt left 19:13 zakharyas left 19:58 sena_kun joined 20:00 Altai-man left 20:02 lucasb left 20:28 brrt joined 20:59 hankache joined 21:26 zakharyas joined 21:36 zakharyas left 22:14 sena_kun left 22:24 hankache left 22:27 brrt left 23:15 tobs joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!