github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:18 lucasb left 00:51 patrickz joined 00:55 patrickb left 01:06 patrickz left 01:10 rypervenche joined 10:51 Altai-man_ joined 12:29 sena_kun joined 12:30 Altai-man_ left
nine MVM_string_utf8_encode_substr feels a bit odd to me. It allocates a result buffer that's 2*length + 4 extra bytes large, which sounds sensible. But when we still run out of space we use a rather generic "lets double the allocation" approach. But can a UTF-8 encode even result in more than 4 bytes per code point? 12:40
moritz no, but more than 4 bytes per grapheme 13:06
nine which doesn't matter as MVM_string_utf8_encode_substr iterates over code points. So the maximum should be 4*length and worst case it's one realloc 13:08
At least we could mark that result_pos >= result_limit MVM_UNLIKELY 13:09
MasterDuke nine : are you working on enabling encode to append to a preexisting buffer?
nine No, since it looks like you're close to being done :) 13:11
MasterDuke ok, just wanted to make sure we weren't both in the middle of it 13:12
nine Most fixed size allocations by count are done by allocate_frame. Most by allocated memory are by bind_key 13:13
timotimo is this your million perl5 object creation benchmark? 13:16
nine yes 13:23
MasterDuke that matches up pretty well with what i see when compiling CORE.c 13:24
14:05 lucasb joined 14:28 Altai-man_ joined 14:31 sena_kun left 15:09 synthmeat left 15:32 synthmeat joined 16:29 sena_kun joined 16:31 Altai-man_ left, sena_kun left 16:54 sena_kun joined
dogbert17 any c experts around? 17:07
I'm just wondering what happens with the variable 'truthvalue' here, is it ever used? github.com/MoarVM/MoarVM/blob/mast...ize.c#L484 17:12
lizmat feels to me there's a break missing after that line ? 17:14
otherwise that line does not make any sense at all
dogbert17 ^^ 17:15
dogbert17 lizmat: I agree, should we wait for a comment fron jnthn as well? 17:16
lizmat I'm 99% sure there's a break missing
dogbert17 ok, I'll add it locally and run a spectest 17:17
sena_kun dogbert17, I think anything from optimize.c (or anything in this area, really) should go directly to jnthn.
dogbert17 sena_kun: indeed but while waiting I can do some local experiments
sena_kun dogbert17, local - sure. :) 17:18
dogbert17 also, shouldn't the compiler output a warning ...?
lizmat no, because absence of a break can sometimes be what you want 17:20
sena_kun lizmat, by the way, you're on osx, right? 17:21
jnthn dogbert17: Looks like a missing break to me. 17:22
nine Yes, in fact the compiler should warn. If it's missing intentionally it should be marked MVM_FALLTHROUGH
Don't know why it doesn't warn in this case. Maybe because it falls through to a default 17:23
dogbert17 thx everyone, unless someone beats me to it I'll send a PR a little later 17:26
jnthn Please do be sure to run tests, 'cus fixing it will enable a further optimization. 17:27
dogbert17 spectest is clean
All tests successful. Files=1306, Tests=111237, 200 wallclock secs (14.12 usr 1.80 sys + 1445.77 cusr 83.55 csys = 1545.24 CPU)
dogbert17 runs a stresstest as well ... 17:28
nine Please run with MVM_SPESH_BLOCKING=1 and maybe MVM_SPESH_NODELAY=1 17:30
spectests are often too short to trigger optimization
dogbert17 nine: will do 17:31
seems to be some kind of cool branch opt 17:40
17:44 zakharyas joined
nine In my leaky example every GC run promotes precicely 2434 bytes to gen2 which grows by 2384 bytes. 17:44
dogbert17 two tests failed: t/spec/MISC/bug-coverage-stress.rakudo.moar and t/spec/S17-procasync/kill.t 17:46
lizmat sena_kun: yes, I'm on MacOS :-) 17:47
dogbert17: isn't t/spec/S17-procasync/kill.t flappy to begin with ?
nine yes 17:48
dogbert17 cool, I'll ignore that one then
nine those are the two I've worked on on Friday
dogbert17 aha, both seems to work when running them manually 17:49
guess I can do a PR then 17:51
lizmat dinner& 17:57
Geth MoarVM: dogbert17++ created pull request #1266:
Add missing break statement		 17:59
MoarVM: 4fb85a017d | (Jan-Olof Hendig)++ | src/spesh/optimize.c
Add missing break statement

For some reason the compiler didn't catch this. Change is spectest clean.		 18:27
MoarVM: 16ff1585e8 | niner++ (committed using GitHub Web editor) | src/spesh/optimize.c
Merge pull request #1266 from dogbert17/fix-break-statement

Add missing break statement
18:28 Altai-man_ joined 18:31 sena_kun left
dogbert17 nine++ 18:31
nine I stumbled over a new insight for my leaky TestMemory.new example: heaptrack will report the object's hash storage as leaked when I don't give the Perl 5 interpreter the chance to clean up on program exit. 18:42
Ordinarily I'd say that this must be a wrong reference count then. But if I add a DESTROY method to that class it actually gets called, a lot. 18:43
Oooooh....this doesn't seem to leak at all: raku -Ilib -e 'use Inline::Perl5; my $p5 = Inline::Perl5.new; $p5.run: "use lib qw(t/lib); use TestMemory"; loop { $p5.invoke("TestMemory", "new"); }' 19:00
Well it does grow...but only by a couple 100K every couple of minutes 19:02
And that means a jump every couple of minutes.
19:03 MasterDuke left 19:13 MasterDuke joined
nwc10 .tell jnthn there is also a Malformed DU chain in t/spec/S32-exceptions/misc2.rakdo.moar -- it's Spesh of 'EVAL' (cuid: 12669, file: SETTING::src/core.c/ForeignCode.pm6:27) 19:38
tellable6 nwc10, I'll pass your message to jnthn
nine Finally! Progress 20:14
Turns out, heaptrack was wrong. It was first wrong about Perl 5 memory not getting leaked and then wrong about which objects get leaked. But it's hardly to blame. Generic tools just have a really hard time with custom memory managers.
The SV that I leaked was the tiny, innocent integer that gets returned by Perl6::Object::destroy indicating whether the object actually got destroyed or resurrected. Needs an sv_2mortal 20:15
timotimo nine: valgrind allows you to tell it about your custom allocators tho 20:16
but also: that's very funny
good catch
nine heaptrack probably just got that wrong because the object gets allocated which triggers a malloc, then the object gets freed and its memory re-used for that integer 20:17
MasterDuke heaptrack also allows your code to tell it about custom allocators according to the readme, but that's all i know 20:22
nine: so this is completely different than the leak you see with the non-Inline::Perl5 example? 20:23
nine MoarVM's heuristic for gen2 collections doesn't help btw. It exacerbates the issue because it compares the promoted bytes to the current RSS. But if RSS keeps growing we will hit the threshold later and later, if at all. 20:24
I think it would be better to take the RSS of the previous full collection as non-moving base line
I'll have a look at that tomorrow or so. So as so often, other users of NativeCall will benefit as well :) 20:26
jnthn nine: Yes, that kind of conern is why I pointed you at it; it wasn't really tuned for a situation where something else is doing lots of allocations that are in the process, but not known to the VM. 20:28
*concern
I'd not thought of using the RSS of the previous collection, that's an interesting idea.
20:29 sena_kun joined 20:30 Altai-man_ left 20:54 zakharyas left 21:15 Kaiepi left 21:16 Kaiepi joined 21:17 Kaiepi left 21:18 Kaiepi joined 21:19 Kaiepi left 21:20 Kaiepi joined 21:21 Kaiepi left 21:31 Kaiepi joined 21:32 Kaiepi left, Kaiepi joined 21:35 Kaiepi left 21:38 Kaiepi joined
nine 2564 nine 20 0 5554236 4.925g 62480 S 0.000 15.73 54:30.19 raku -Ilib -I/home/nine/Inline-Perl5 service.pl6 22:08
2602 nine 20 0 3097392 1.945g 67168 S 0.000 6.215 24:04.08 raku -Ilib -I/home/nine/Inline-Perl5 service.pl6
That's 1G off the first one and tens of megabytes off the frontend
MasterDuke nice! 22:09
22:28 Altai-man_ joined 22:31 sena_kun left 22:35 lucasb left
timotimo phew. 22:46
22:55 Altai-man_ left 23:15 rba[m] joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!