github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
MasterDuke jnthn, timotimo: i'm looking at a perf report (of slurping a file with Compress::Zlib) where the top function at 45% (next at 7% is MVM_interp_run) is sim_stack_find 00:14
any idea what could be causing that? 00:15
00:16 Kaiepi joined
jnthn MasterDuke: No, though that doesn't sound good 00:17
Is the program somehow heavily recursive?
If not, that sounds especially suspect
Even if it is, it's still suspect at 45% 00:18
I mean, it's off on another thread, but that's still pretty bad.
MasterDuke not recursive at all i believe 00:19
jnthn Weird. Could you open a MoarVM issue? 00:20
MasterDuke it causes a noticeable slowdown too. runtime is ~9.2s usually, but ~14.5s when that happens
jnthn With repro instructions
MasterDuke sure
jnthn I'd like to investigate that. Probably won't get to this side of the German Perl Workshop, though.
MasterDuke any further information i could gather? 00:21
jnthn Well, spesh log is a bit big to post I guess
And might not give many hints anyway
And if there's repro info, it's easily product 00:22
*produced
Spesh "replays" the execution to model the call stack
Something seems to be going terribly wrong in that
How is memory use, BTW?
You might also run it under callgrind to see if that gives the same results 00:23
00:23 nebuchadnezzar left
jnthn Just in case there's anyone here who it applies to, twitter.com/jnthnwrthngtn/status/1...4172324864 00:23
MasterDuke memory use is about the same in either condition
00:23 nebuchadnezzar joined
jnthn That's stranger still...I'd have thought it'd be building/searching a huge stack 00:24
MasterDuke 197412maxresident in fast case, 191928maxresident in slow case
00:25 Kaiepi left 00:28 patrickz joined 00:31 patrickb left
MasterDuke jnthn: github.com/MoarVM/MoarVM/issues/1060 00:36
jnthn Thanks 00:40
MasterDuke jnthn: on a different, but related, topic is gather/take as fast as it's going to be for the forseeable future? 00:50
jnthn Hm, interesting. The biggest win was making it not take the continuation in the eager case, which I think if you're asking for .elems should be the case. It may be worth checking that is really happening. 00:53
In the lazy case of course it has to do that 00:54
Taking/invoking that only got one notable speedup recently, which is that after dropping the ref-counting it didn't have to walk the frames during invoke 00:55
And that's recent as in, some years ago :)
There may be a possible optimization in caching exception handler lookups 00:56
I'm not quite sure how we'd factor that, but it'd maybe be worth something if we didn't have to scan the stack for a take handler if it's deep-ish 00:57
Might be a way to save something on the way we store the continuation tag (avoid a mallc or something) 00:58
Though that may not be worth much at all
MasterDuke jnthn: how would i check that it's not taking the continuation in the eager case?
jnthn Hmm...if the allocation profiler is accurate in this regard, look for continuation allocations in that 00:59
Though it maybe ain't. Hm. 01:00
Otherwise there's always sticking some instrumentation into continuation.c :)
MasterDuke 100,000 BOOTContinuation allocated
i can look into continuation.c also 01:01
jnthn Ah, cool, then it is logging them :) 01:02
And 100,000 sounds like a lot of it is happening
MasterDuke that's exactly the number of lines in the decompressed file
oh, also 100,006 BOOTException allocated 01:03
jnthn Interesting. 01:06
If you .eager.elems is it different?
MasterDuke and 100,001 Encoding::Encoder::Builtin. does one of those really have to be allocated for every line?
jnthn Um...I sure hope not! 01:07
But that sounds more likely something the module is doing, perhaps?
MasterDuke probably. it looks like it sets an encoding once, but i'll look into that also 01:09
.lines.eager.elems didn't make a difference
same runtime, same number of BOOTContinuations
jnthn OK, that's worth looking into 01:12
Bed time for me, 'night o/
MasterDuke thanks for the help, later...
01:49 MasterDuke left, MasterDuke joined, MasterDuke left, MasterDuke joined
MasterDuke timotimo: you around? it looks like every time Str.encode() is called it allocates a Encoding::Encoder::Builtin. does that have to be the case? 01:52
timotimo oh, i'd expect the encoding registry to give the same object every time if the encoding is the same? 01:54
but i don't know much about the encoding registry business
MasterDuke github.com/rakudo/rakudo/blob/mast....pm6#L2280 01:55
github.com/rakudo/rakudo/blob/mast...m6#L64-L71
02:02 robertle left, robertle joined 02:14 patrickz left 02:24 Kaiepi joined
MasterDuke .ask jnthn every time Str.encode() is called it creates a new Encoding::Encoder::Builtin. does that have to be the case? github.com/rakudo/rakudo/blob/mast....pm6#L2281 and github.com/rakudo/rakudo/blob/mast...in.pm6#L29 04:18
yoleaux MasterDuke: I'll pass your message to jnthn.
05:49 robertle left 07:31 domidumont joined 07:33 domidumont left 07:34 domidumont joined 07:37 domidumont left 07:38 domidumont joined 10:02 domidumont left 10:29 domidumont joined 14:27 lucasb joined
timotimo re-using an encoder object is An Optimization Technique i guess 14:29
imagine how bad it is on windows with $translate-nl turned on, it'll build twice as many objects 14:31
14:33 brrt joined
jnthn One could also address it in the module by using the encoding registry to get the encoder and then just re-using it each time 14:35
yoleaux 04:18Z <MasterDuke> jnthn: every time Str.encode() is called it creates a new Encoding::Encoder::Builtin. does that have to be the case? github.com/rakudo/rakudo/blob/mast....pm6#L2281 and github.com/rakudo/rakudo/blob/mast...in.pm6#L29
09:54Z <patrickb> jnthn: Could you type together a project idea in the ideas repo for the dynamic optimizer idea you tweeted about? That should hopefully greatly increase the visibility of that idea for students. (github.com/perl-gsoc-2019/ideas/bl...ing-ideas)
09:54Z <patrickb> jnthn: It's great you're willing to mentor!
brrt ohai
dogbert21 hi brrt 14:37
brrt hi dogbert21 14:38
dogbert21 timotimo: were you looking at the bug in t/04-nativecall/06-struct.t, i.e. a problem which seems related to arrays and inlining?
brrt: how is your jit work progressing? 14:39
timotimo, nine: this bug gist.github.com/dogbert17/0427f05e...30878e7091 14:43
timotimo that's the kind i'm looking into, yeah 14:46
dogbert21 cool, then I'll go back to sleep :) 14:50
timotimo at least now i've found the right spot to look at actually 14:51
dogbert21 ++timotimo 14:52
timotimo MVM_nativecall_make_cstruct or _make_carray are called with nothing to signal whether they are inlined or not 14:54
so it just puts the pointer in and forgets about it
later on, the CArray or CStruct that's inside the other just blindly calls free on its pointer, which is, as asan says, not actually an address that came from malloc 14:55
dogbert21 timotimo: do you have any ideas about how to fix it? 15:25
timotimo yup 15:31
i'm on it
i think we might want to have a way to annotate CStructs coming back from C being either "free when the p6 object gets destroyed, or don't free" at the signature level 15:32
kind of like explicitly_manage i guess
i was just about to put in code saying "if !inlined set owned to 1", but that's not quite the right wording, as the C land may have actual ownership 15:33
also, nativecast can really screw us up here; it'd probably always have to set "do not free this" 15:34
however
when you nativecast something from one struct to another, and you're not keeping the original object around, then you can actually end up in a situation where the actual memory of the struct gets freed (because the old version died) but you're still using it (via the "new" cstruct object)
OTOH, if you just nativecast to a different type to set one field and then continue using the other cstruct, that'll have the opposite effect; your old object has to stay responsible for the lifetime, and the new object will immediately become garbage and mustn't free the pointer 15:35
isn't all this very fun?!
requiring the user to always call free on CStructs or leak memory is also shite, but as it stands i believe we're just never freeing CStruct objects' data pointer ever? 15:37
unsurprisingly, the code with the commented-out MVM_free call says "for some reason this causes crashes at the moment" 15:38
so i guess whatever i do it'd be an improvement?
brrt dogbert21: slowly 15:45
I've found a fairly interesting bug in the expression optimizer 15:46
dogbert21 brrt: is that a bug which affects us today? 15:52
timotimo: sounds quite messy
timotimo isn't it 15:53
maybe a little syntax brainstorming session could be useful here
like, i could imagine nativecast(MyOtherStruct, $value, :transfer-ownership) or something could be fine
i could also imagine having an extra object that defines the lifetime of an underlying c pointer and that would be stored in the CStruct or CArray or whatever objects 15:54
when that dies, MVM_free is called
if you first call .disown on it, MVM_free is never called, for example when you give ownership over to C
and maybe .reown could be A Thing
inlined structs and arrays would get a reference to the parent object's ownership object 15:55
AlexDaniel samcv: ETA, if possible? I wanna plan my time a little 16:08
16:13 brrt left
dogbert21 hmm 16:21
timotimo so, i think there would have to be new ops for handling the ownership of strings 16:29
since the reprs we use dibt support attributes that are not part of the native memory
but on the p6 api side, the ownership management ops could totally live in the metaibject 16:30
16:33 domidumont left
timotimo $myobj.^disown 16:46
doesnt sound so bad
i had that already typed out but there was a hiccup in ssh due to mobile neteirks and weechat thoughz ibwas pasting 16:47
anyway, this new ownership concept could be the basis of a unified thing that also covers strings i guess?
though i wonderif it will be necessary for each object to remember if it was the "originator" of the thing or not on top of sharing ownership possibly 16:48
and if there should be an interface based on the ownership thing that can also kbow tze size of things for the cases where we actually kbow 16:49
it would allow writing to CStr objects i think
though ibwouldbhave to look at the CStr repr again 16:50
ok cstr does remember its original pointer already 16:53
samcv AlexDaniel, will take a little longer than I had expected 17:00
so don't wait for me 17:01
17:09 domidumont joined
AlexDaniel šŸ‘ 17:11
17:29 patrickb joined 17:41 robertle joined 17:48 squashable6 left 17:54 reportable6 left 17:58 reportable6 joined 17:59 shareable6 left, nativecallable6 left, committable6 left, unicodable6 left, releasable6 left, greppable6 left 18:00 squashable6 joined 18:01 committable6 joined 18:03 shareable6 joined
timotimo on the road i was thinking maybe CStruct and CArray could be parameterized for "will be owned by C" or "will be owned by moar" so it could be put into signatures 18:16
but then i was thinking that'd mean type checking would have to be a bit more complicated, because they'd have to be considered the same type even when that parameterization differs
for instance, if you have the situation that i have right now with SDL2::Raw where there's a void **foo that'll take an initialized memory location to store a pointer to a buffer into, i'd offer a CArray that's owned by moarvm and inside there'd now be a CArray that's owned by C 18:43
18:49 brrt joined
nine I can't help but feel reminded of the musings I did on explicitly-owned strings and NativeCall 18:49
19:20 brrt left 19:26 nativecallable6 joined 19:30 domidumont left 19:46 robertle left 20:18 unicodable6 joined
timotimo nine: where can i find that? just in the irclog? 21:11
nine timotimo: maybe also some ticket, though I can't remember anything exactly 21:13
colabti.org/irclogger/irclogger_lo...-09-01#l38 21:15
Ha, I even noted that some of it would apply to other allocated types beside strings :) 21:17
Always fun to discover your own work: gist.github.com/niner/53c4da135580...05a1d7895b
21:19 Kaiepi left 21:20 Kaiepi joined
timotimo now i'm wondering how my idea with ownership objects would relate to string constants 21:23
it wouldn't really be on Str, only on CString
i'm getting the feeling that the ownership idea fits very nicely with CStruct/CArray, but breaks down a bit when applied to CString 21:25
since with CString you often want to just put the literal string in places and have everything work out magically 21:26
and i don't think i have a way to handle when C land gives back a pointer that actually already has an ownership object somewhere else 21:30
and if C land gives back a pointer that is actually a pointer into a different object ... 21:31
22:18 sivoais_ left 22:19 sivoais joined
MasterDuke jnthn: but encode() doesn't take an encoder. the module would have to know how encode() is implemented and do that itself manually. which is obviously possible, but is that what they have to do if they want to be more efficient? 22:33
jnthn, timotimo: should we just create the encoders most likely to be used (e.g., utf8, strict/non-strict) at rakudo compile time? 22:35
22:50 releasable6 joined
jnthn MasterDuke: Yes, the intention was that the Encoding API would actually be used for such cases 23:03
I'm a little wary about caching in that encodings may be configurable with different replacements and so forth 23:04
I guess it's even feasible that an encoding could be stateful 23:05
23:14 patrickb left 23:41 greppable6 joined
MasterDuke jnthn: i was thinking a very small cache. or maybe not even caching "new" encoders, but just pre-creating a common couple ones 23:54
hm, Encoding::(De|En)coder aren't documented 23:57

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!