colomon MoarVM near the top of HN at the moment! 02:58
JimmyZ HN? 03:00
colomon Hacker News 03:07
TimToady is this good or bad? 03:08
colomon news.ycombinator.com/news
[Coke] looks like there's no real thread talking about it, just a linklist. 03:15
moritz 08:41 < arnsholt_> MoarVM on the HN frontpage! news.ycombinator.com/item?id=7357670 08:14
yawnt hi :P 10:11
FROGGS hi yawnt 10:13
JimmyZ timotimo and anyone: github.com/dtrace4linux/linux :) 13:23
timotimo oh, not bad 13:25
TimToady my @bar = (gather take 42) while 1; 17:48
that is a very fast leak on maorvm
jnthn Mebbe worth valgrinding... 18:03
isBEKaml OHHAI, I ran into this error, when I tried building MoarVM on Linux( Slackware - perl v5.10): sprunge.us/KGhQ
does moarvm require newer perl versions?
jnthn Not sure we ever laid down a requirement so far... 18:04
TimToady has never valground, are there any experts in the house 18:05
jnthn Grrr...train is rocking a little too much on this section of hte journey for me to be able to keep looking at a screen without feeling ill :/
SJ, why you no lay track flat...
isBEKaml jnthn: I don't think it's really much to do with newer perl as much as asking for a newer perl module (File::Path, in this case). 18:07
it's build/probe.pm asking for unexported/absent subs in File::Path. 18:09
jnthn isBEKaml: OK. Probably whoever wrote the code had a newer version. Then, on this box I've a stock 5.16...
...with no updated modules I know of. And it's working.
nwc10: Dunno if you know, given I think you worked on probe code?
oh dears... 18:10
src\strings\ops.c(1390) : error C2275: 'MVMCodepoint32' : illegal use of this type as an expression c:\consulting\moarvm\src\6model/reprs/MVMString.h(4) : see declaration of 'MVMCodepoint32'
dalek arVM: 3787171 | jonathan++ | src/strings/ops.c:
Unbust MSVC build.
 18:12
isBEKaml Ah, it's mkpath and rmtree in my version of File::Path
Why was the module's API changed? :/ (Modifying build/probe.pm to use mkpath and rmtree got me going, for now) 18:15
jnthn TimToady: Reproduced the leak. Maybe I'll find it. 18:20
TimToady it's a full fart leak :) 18:22
isBEKaml Alright, he's experiencing that in a train :-) 18:22
TimToady I daresay the leak is going faster than his wifi... 18:23
isBEKaml metacpan.org/pod/File::Path#API-CHANGES -- make_path and remove_tree use mkpath and rmtree under the hood. Can we not detect versions and fallback accordingly? 18:28
jnthn Well, I found a leak, but not the leak... 18:47
TimToady cool, find a bunch moar :) 18:48
lizmat
.oO( Hans Brinker to the rescue )
 18:50
TimToady is possible it's a space leak instead, perhaps in the eager implied by sink, if it doesn't actively throw stuff away as it's generated 18:53
nwc10 I hadn't realised how "new" make_path and remove_tree are 19:05
fix is to rework the code to use the older functions
I certainly don't have time today
I won't on Sunday
I may not tomorrow either
it's a SMOP for anyone who knows Perl 19:06
isBEKaml nwc10: I don't :-)
nwc10 most of next week looking unlikely
isBEKaml nwc10: It's not a big deal anyway. I got rakudo-m built and ready after changing probe.pm to use mkpath and rmtree. 19:12
nwc10 yes, but I think we should patch the probe code 19:38
jnthn OK, I managed to get a build with the CRT debug malloc that can do leak detection. 19:45
FROGGS coool
TimToady I'll bet my leak is "special". :) 20:04
but I'd be glad to lose that bet...
jnthn Trouble is, we've not been too good at making sure stuff is freed at exit 20:08
So there's a ton of leakage
(Knew I'd need to take it on at some point... :))
dalek arVM: efa454b | jonathan++ | src/6model/reprs/MVMContinuation.c:
Fix leak of active handlers in continuation.

If a continuation is never resumed but it saved handlers, they would never become active again and would thus be leaked.		 20:11
arVM: 46000f6 | jonathan++ | src/gc/gen2.c:
Clear up gen2 memory at exit.
TimToady p5 is very picky about cleaning up when it knows it's embedded 20:18
if it's not embedded, it only cleans up enough to attempt to run all DESTROYs
jnthn Well, I'd rather cheat once we've learned to not cheat :) 20:19
TimToady since exit is a rather efficient way to clean up the rest of 'em
ayup
jnthn I'm gonna go through this list and zap the easy ones, to try and work out where the real ones are. 20:20
dalek arVM: 4711808 | jonathan++ | src/6model/reprs/MVMStaticFrame.c:
Don't leak static frame instruction offsets.
 20:23
arVM: e62d7c7 | jonathan++ | src/core/ (3 files):
Clear up frame pool at exit.
 20:49
jnthn Somehow, hashes leak somewhat. 21:00
dalek arVM: b07e028 | jonathan++ | src/6model/reprs/MVMHash.h:
Avoid leaking 72 bytes per MVMHash.

In gc_free for MVMHash, due to HASH_CLEAR clearing the head pointer, we ended up never freeing the memory for the first node in a hash.		 21:21
jnthn TimToady: So, having cleaned up this much, I can confirm that we seem to be leaking frames. 21:22
TimToady: Hm, but it doesn't seem to be enough to account for the loss 21:25
ooh, nearly home... 21:26
bbiab
lizmat hopes jnthn got home safely 22:55
timotimo hm. i wonder if the leak reduction for mvmhash will cause better startup memory usage yet 22:57
jnthn lizmat: Yeah, just a bit wet :) 23:01
timotimo: tits
tadzik what
jnthn try it to see 23:02
:P
tadzik :D
jnthn timotimo: It will probably reduce CORE.setting comp memory
timotimo: 'cus we allocate and throw away a lot of hashes.
timotimo that's very good. 23:05
i'll try it out, actually
jnthn Yeah, I'm just doing a build here too 23:07
timotimo 89852maxresident - not really seeing a difference here 23:11
jnthn Pretty sure CORE.setting build was lower though. 23:12
timotimo oh, that
should verify.
jnthn yeah, I only eyeballed it.
timotimo i'm verifying it now 23:13
what revision should i revert moar to?
Unbust MSVC build. ← this one?
wowza. 620 megabytes of maxrss for setting compilation 23:14
absolutely speaking, that's quite a lot of memory 23:15
relatively speaking, that's probably way better than parrot
oh wow. can this be real?
67.68user 0.28system 1:08.20elapsed 99%CPU (0avgtext+0avgdata 742696maxresident)k
70.31user 0.22system 1:11.00elapsed 99%CPU (0avgtext+0avgdata 619296maxresident)k
timotimo collects more datapoints 23:16
jnthn Yes, unbust is the one. 23:17
Hm, is that suggesting we went from 743 to 620 with my fixes? :)
timotimo yes 23:18
jnthn wow.
timotimo so leak
much garbage
jnthn so fix
timotimo at least the memory usage seems pretty stable in the "before" version 23:19
the numbers seem to be legit 23:23
good catch!
jnthn does himself a leak detecting build on his main dev machine 23:25
timotimo the memory consumption numbers are quite stable 23:26
there's also leakage happening caused by string decoding and/or encoding according to valgrind 23:28
want me to run the analysis?
i wonder how long the core setting takes when running under valgrind
since we do our own allocations in our own heaps for a big chunk of all allocations, it may not be terribly slow compared to normal programs that use malloc for every little thing 23:29
jnthn timotimo: I see them here too, so no need to valgrind unless you also want to go hunting. :) 23:30
timotimo oke 23:33
í'll do it just to find out how much longer it takes
hm. usually like 20x longer, right?
jnthn no idea :) 23:35
TimToady: With the cruft cleared up, it's fairly clear we're leaking frames in the gather/take, which is why your example swallows so much memory. 23:36
TimToady yay! 23:37
jnthn mmm...rochefort 10...
[Coke] so glad I get to program in perl6 and not c when jnthn is done. :) 23:38
jnthn Only question now is why...
timotimo why rochefort? 23:40
TimToady [Coke]: well, I still note that the narcissist code equivalent to that which runs hours in Perl 6 ran in about .3 seconds in D, so we still have a ways to go...
jnthn timotimo: 'cus it tastes nice! :P 23:43
timotimo heh
hm. now i have to go. i can't suspend the laptop during the run because that'd give me wildly exaggerated measurements 23:44
ctrl-c'd it and got these numbers: ==8728== definitely lost: 3,584,385 bytes in 98,440 blocks
==8728== indirectly lost: 1,226,998 bytes in 9,340 blocks
==8728== possibly lost: 54,773 bytes in 119 blocks
probably just means moarvm doesn't bother cleaning up when you ctrl-c it 23:45

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!