00:00 geekosaur joined 00:12 zakharyas joined
ugexe why is it not necessary to MVMROOT async_type here? github.com/MoarVM/MoarVM/blob/0441...ers.c#L109 01:01
01:55 ilbot3 joined 03:33 dogbert2 joined 04:29 geekosaur joined 04:41 geekosaur joined 04:45 travis-ci joined
travis-ci MoarVM build errored. Jonathan Worthington 'Merge pull request #679 from ugexe/patch-1 04:45
travis-ci.org/MoarVM/MoarVM/builds/273397039 github.com/MoarVM/MoarVM/compare/e...322f393365
04:45 travis-ci left
JimmyZ ugexe: when: MVMOject *a; a function may trigger GC, use a. and then you need MVMROOT(a) arround 'a function may trigger GC' 07:05
ugexe: in that exmaple, async_type is not used after 'a function may trigger GC' 07:06
ugexe: 'a' must be NULL or MVMObject 07:07
s/GC, use/GC; use/ 07:09
07:19 robertle joined 07:31 domidumont joined 07:38 domidumont joined 08:08 BooK joined 09:10 robertle joined 10:39 Skarsnik joined 10:51 Skarsnik_ joined 11:11 dogbert2 joined 11:17 leont joined
dogbert2 Good afternoon #moarvm 11:34
I might have asked this before but if I have then I have forgotten the answer
there's an --optimize switch in Perl 6, what kind of optimization does it actually do? I guess it has nothing to do with either Spesh or the JIT. 11:36
jnthn Some lexical -> local lowering, some scope flattening, various peephole-y rewrites (like if $a == 1 | 2 { } into something cheaper using a temporary and ||), and inlining of basic native operators 11:39
Skarsnik hm this get called to free the memory? github.com/MoarVM/MoarVM/blob/mast...ray.c#L121 11:40
jnthn Yes 11:41
dogbert2 jnthn: thx, so it deals with stuff which does not need any runtime analysis
jnthn Of note, it's called by the GC when the object has become unreachable
gc_free is for top-level objects, gc_cleanup is when it was embedded into another object 11:42
dogbert2: Correct. And we try not to put too much there, because for quick scripts without any hot loops in them it's mostly wasted effort.
See slides from my SPW talk on deopt for a little more about that 11:43
BooK jnthn: regarding github.com/MoarVM/MoarVM/issues/683 I'm shying away from working on a patch because of the probable signature change needed
Skarsnik I am curious because valgrind repport a leak on the expand function in this, this seems true since it always allocate stuff for the child_objs in the body but they will not get free if body->managed is false 11:44
11:44 travis-ci joined
travis-ci MoarVM build errored. Jonathan Worthington 'Merge pull request #671 from duke-m/patch-1 11:44
travis-ci.org/MoarVM/MoarVM/builds/273397310 github.com/MoarVM/MoarVM/compare/3...735a899891
11:44 travis-ci left
dogbert2 jnthn: many thanks 11:46
timotimo oh, i have been meaning to look into whether the junction optimization is currently working
Skarsnik I am not even sure expanding a C managed array is a good idea in the first place x) 11:47
jnthn BooK: Signature change to a MoarVM op?
timotimo aha, it's broken. as i feared. 11:48
jnthn BooK: One option is to introduce a new op (take one of the oldest DEPRECATED_n)
timotimo dogbert2: you can look in rakudo's repo under Perl6/Optimizer.nqp (and there's also a regex optimizer)
jnthn BooK: And map the NQP one to the new one, and then we can deprecate/remove the old one later 11:49
Ah, I forgot the regex optimizer :)
timotimo the regex optimizer ought to do a whole lot more work, as we can't do much there in a dynamic optimizer 11:54
Skarsnik hm what gc_mark does? register memory to the gc? 11:57
timotimo it registers pointers
because the GC has to update pointers when the target objects move
but the pointers also keep other objects alive 11:58
ah, ok. == handles Mu, so i'm not allowed to do the static optimization of unfolding the junction for 'em 11:59
dogbert2 timotimo: do you know if --optimize=3 is the default? 12:01
jnthn No, 2 is 12:02
Skarsnik Ok, so there is memory lost in CArray. If I understand correctly : Since it does not know his own size, it will call expand (via at-pos) when going through it (when using a C for loop). This will allocate object for the perl6 repr of the content? when the Carray come from a C call it does not allocate the C memory for it. But when a gc_free is called on the array it only remove the perl6 obj if Moar has the hold of the Array (memory wise). So the perl6
obj created by expand are lost when it's managed by the lib
timotimo 3 is set explicitly for the core setting
dogbert2 so it (--optimize=3) shouldn't be used for 'normal' p6 scripts then 12:03
timotimo you can use it if you want to
jnthn Skarsnik: Sort of. The memory that points to the Perl 6 objects created by expand would be lost, I think. 12:04
dogbert2 I bet there's a downside somewhere :)
jnthn Skarsnik: The objects themselves would not be marked, however, and so could be collected
timotimo it looks like 2 and 3 are currently equivalent
Skarsnik and I am not sure how to fix that?
jnthn Skarsnik: Since our GC is about reachability, not ref-counts
Well, if it's just as you described it, would the fix not just be to unconditionally free child_objs? 12:05
Just move it outside of if managed
iirc it's just a piece of C memory that points to other MoarVM objects, and the GC will take care of those objects 12:06
Skarsnik The issue I am not sure what these obj are. like if they contains other stuff that came from the C lib?
jnthn Maybe, but the array you'd be freeing, that I agree we seem to be leaking, is *this* object's list of pointers to the other objects 12:07
We'd not be changing the lifetime of the other objects by freeing that array
dogbert2 speaking of optimization, is it still the case that array indexing is less optimized than it could be?
jnthn Since it's just a piece of malloc'd memory, not something that our GC tracks 12:08
dogbert2: Yes
timotimo yeah, i was able to get much better performance from .AT-POS than from .[]
even in the simplest case
jnthn Inlining should make the difference about go away, but until we get the derived type specializations and then spend some more time tweaking and tuning...
dogbert2 a lot more hard work then .. 12:09
timotimo jnthn: that's when we see that a part of the type tuple is static and another part is megamorphic? 12:10
Skarsnik jnthn, Thanks, I am not used to working with a GC ^^ 12:11
jnthn timotimo: Yes 12:12
timotimo ugh. i'll have to be much more clever in finding candidates for multis that handle Mu 12:13
i'm not sure if i can properly reuse the "find me a multi that matches" code
Skarsnik jnthn, spesh seems to lost lot of memory according to www.nyo.fr/~skarsnik/tmp/valgrind.txt x)
jnthn Skarsnik: Hard to say 12:14
It is running in the background, and when the process is torn down we don't try and sync up with the spesh thread and get it to clean up 12:16
So worth a look but it could just be that that's the working memory it happens to have at the time of shutdown
Hm, the later parts of it do look a lot more suspect though 12:17
MasterDuke somewhat related, here's a heaptrack summary of a script that essentially just calls Proc::Async.new 5 times, capturing the output of 'git rev-list <options>': imgur.com/zD2CfJm
Skarsnik dang this heaptrack thingy look interesting 12:18
MasterDuke some more info is in a gist linked from github.com/MoarVM/MoarVM/issues/680 12:19
jnthn Yeah, looking more closely it seems spesh is losing something
MasterDuke (i ran the script with fewer iteration than in the gist for the heaptrack record)
oh, that heaptrack record may have had MVM_SPESH_DISABLE=1 set 12:20
here's without that: imgur.com/lhqm6HS 12:23
timotimo cool, my fix works 12:24
now to spec test
i bet this has been broken for months now :S 12:25
ever since all these ops got a Mu $?, Mu $? signature 12:26
Skarsnik the NQPMu stuff?
timotimo no
&infix:<==> has a proto sub &infix:<==>(Mu $?, Mu $?) is pure { * }
the junction optimizer opt can't statically do anything if the sub accepts Mu, because then it wouldn't autothread 12:28
i just changed the opt to only look at candidates if the sub is a proto sub
12:34 Ven`` joined 12:37 domidumont joined
Skarsnik if someone want to push these: www.nyo.fr/~skarsnik/tmp/various-nc-stuff.diff ~~ 12:52
13:15 travis-ci joined
travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #679 from ugexe/patch-1 13:15
travis-ci.org/MoarVM/MoarVM/builds/273397039 github.com/MoarVM/MoarVM/compare/e...322f393365
13:15 travis-ci left
Geth MoarVM: Skarsnik++ created pull request #684:
Fix a leak in CArray repr.		 13:27
13:27 leont joined
Skarsnik this edit file directly on github is handy ^^ 13:27
timotimo yup! 13:34
13:49 travis-ci joined
travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #671 from duke-m/patch-1 13:49
travis-ci.org/MoarVM/MoarVM/builds/273397310 github.com/MoarVM/MoarVM/compare/3...735a899891
13:49 travis-ci left
AlexDaniel handy, by doesn't sign your commits (obviously) 13:55
same with PR merges
but*
timotimo i haven't done any commit signing :S 13:58
AlexDaniel timotimo: you set it up once and that's it :) 14:07
if you sign your mails already then I guess you don't even have to create a gpg key… 14:08
timotimo i used to have a good gpg key with lots of signatures 14:26
dogbert2 if a program runs slower with the current release, compared to say 2017.07, is callgrind the tool to use in order to figure out why? 15:02
15:47 robertle joined 15:55 zakharyas joined
BooK so regarding github.com/MoarVM/MoarVM/issues/683 jnthn said I could introduce a new op using an oldest deprecated one, and I didn't get fully what that meant 16:00
if someone is interested in that issue, and willing to take me by the hand implement my proposed change, I'd be delighted
think of it as an investment ;-) 16:02
Skarsnik BooK, for this issue, resolve_blabbla is not exposed at nqp/rakudo level 16:05
it's only exist for connect and listen (and another one maybe) maybe looking at github.com/MoarVM/MoarVM/pull/659 can help you figure this stuff. What is probably missing is getting the sa-family attribute at Moar level 16:08
BooK Skarsnik: no, it's not, but the extra argument needs to be passed from Perl6 16:13
Skarsnik passing the io:s:inet obj could probably be more useful x) 16:14
BooK regarding that 659 change, I was surprised it was needed. after all, a network connection is defined/identified by these four parameters 16:15
Skarsnik: can it be passed down as is to the vm layer? 16:16
Skarsnik other wise it became ridiculous nqp::connect($host, $port, $source-host, $source-port, $dest-sa, $source-sa)
BooK sa is sockaddr?'
Skarsnik sa-family I mean 16:17
BooK well, there's only one socket, so indeed it could just be the full object
Skarsnik or whatever the name at Rakudo level x)
BooK yeah, family is a socket attribute, so just one's needed
but it's true that all these args are related to the socket, so maybe just pass them all at once 16:18
Skarsnik yeah I am dumb. sa-family must be the same on both side anyways
BooK otoh, a port of 0 is not the final version of it, it's just saying "I don't care, let the system pick it"
Skarsnik Yes
BooK so it's not the actual value for the socket once it's created
Skarsnik but passing the objet could be more useful, like to set some attribute, like port choosen, ip of the host or stuff like this 16:23
BooK is it what your getport does? getting the port chosen?
Skarsnik getport? i think it's another pr, not from me ^^ 16:24
BooK oh sorry
Skarsnik but I think that what it does
BooK so it wouldn't be needed anymore
the ops that seem to need to be modified to receive the socket obj are: socket, connect, bindsock 16:25
not sure about accept
Skarsnik I think io::s:inet was rushed because I think the async version is prefered x)
BooK but at that level there sure must not be that different 16:26
in the end, it's still the same syscall that creates the socket
Skarsnik No, async socket use libuv
very different code
BooK ah ok 16:28
so I've tried to follow bindsock, and I think that led to bind_sk, and then to MVM_io_bind is that right? 16:30
(from nqp to moarvm)
and so all of those have to change is MVM_io_bind is to receive the IO::Socket::INET object from Perl6 16:31
is GET_REG(cur_op, 0).o an object? 16:33
Skarsnik I am just not sure at all nqp accept fully build Perl 6 object. should ask in perl-dev maybe x)
Try to see if there is an op that take an object? 16:34
BooK in moar? 16:36
Skarsnik dunno, in nqp or Rakudo? 16:37
timotimo um, having an IO::Socket::INET object from perl6 seems like a very bad idea 16:38
we have nqp::socket for a reason
you should be using that object to set low-level attributes and such 16:39
BooK ok
16:39 travis-ci joined
travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #631 from MasterDuke17/smarter_on_alloc 16:39
travis-ci.org/MoarVM/MoarVM/builds/273399542 github.com/MoarVM/MoarVM/compare/1...0f8972b27c
16:39 travis-ci left
BooK timotimo: I only ever looked at moar yesterday, so I'm pretty much don't know anything 16:40
hence my call for some hand-holding
timotimo right 16:42
in the IO::Socket::INET code you can see the $PIO and $!PIO variables being passed to nqp ops
BooK yes 16:43
timotimo that's where the real communication with moarvm happens
BooK ok
my $PIO := nqp::socket($.listening ?? 10 !! 0); # so nqp::socket returns a new nqp/moarvm socket object? 16:44
timotimo MVMOSHandle is one interesting thing to look at, in moarvm's 6model/reprs folder
that's right
look for MVM_io_socket_create in the source to see how it's created 16:45
in syncsocket.c
BooK and $!PIO is a private attribute that keeps that lowlevel socket object
?
timotimo yes
BooK so shouldn't all this family stuff we're talking about already be in the low-level socket? 16:46
timotimo you might want to give MVMIOSyncSocketData a field to hold the address family or something 16:47
well, that lowlevel socket object is still one layer above the regular socket api you might know
you'll see the MVMIOSyncSocketData has a member called "handle", that's the one the OS wants to have
though perhaps putting the address family into the struct is unnecessary 16:48
depends on what OS-level functions want to have the address family value
ok, gtg
BooK it's all in MVM_io_resolve_host_name I think 16:50
the family is needed to give better hints to getaddrinfo
Skarsnik But this function only get the thread context and the hostname/port ^^ 16:53
BooK and MVMIOSyncSocketData is built on the already existing socket, I think 16:59
Skarsnik: also, changing MVM_io_resolve_host_name is probably not a big issue 17:02
Skarsnik yeah it's only used in Moar
BooK socket_connect gets a MVMOSHandle
Skarsnik that sucks 17:03
BooK does it?
Skarsnik well if it does not get the syncsockdatastruct or whatever is the nqp::socket objet you have to change the nqp instruction 17:04
BooK it's defined here github.com/MoarVM/MoarVM/blob/mast....h#L13-L16 17:05
Skarsnik oh nvm it's MVMIOSyncSocketData *data = (MVMIOSyncSocketData *)h->body.data; 17:06
17:06 domidumont joined
BooK right 17:06
so is there anything in MVMOSHandleBody / MVMIOSyncSocketData we can use?
gtg but I'll definitely be back :-) 17:07
Skarsnik ^^ 17:08
ugexe JimmyZ: so you don't have to root things that are passed -to- the function, only things used after it? 17:19
JimmyZ ugexe: yes 17:21
ugexe JimmyZ: thanks for explaining 17:23
JimmyZ ugexe: the function will MVMROOT it inside if it has something gc trigger inside
ugexe: which will used after another gc trigger function 17:25
17:53 Skarsnik_ joined 18:43 ggoebel joined
Skarsnik I wonder if someone ran the whole spec test in valgrind x) 20:38
timotimo that'd take a week or so 20:41
not to mention you'd get some test failures from tests that expect threads to generally execute at the same time 20:42
Skarsnik I mean each separate test file
timotimo me, too
Skarsnik I kinda want to try now
I mean tomorrow
timotimo you should start it now so it can run while you're sleeping 20:50
Skarsnik there is already a process running with efence that take 100% cpu and 70% of the 4Gb of ram x) 20:52
timotimo OK 20:53
poor you with 4 gigs of ram 20:54
i know what that feels like now
(it's why i have 8 gigs of zram)
leont Yeah, I have 8 in my laptop, and sometimes still run out of it 20:55
16 in my work laptop, so I can actually run VMs 20:56
timotimo i used to have 12, then i b0rked it 21:03
Skarsnik I have 8Gb on my desktop 21:05
I use my vps to run long thing
timotimo ah 21:06
Skarsnik the issue I have chrome that take like 4gb alone xD 21:10
timotimo yeah
i also suffer from that
and if i open thunderbird in addition, bam.
Skarsnik I often have to start a 3d game to have him release memory ^^ 21:11
22:03 dogbert2 joined 22:04 zakharyas joined 22:17 zakharyas joined 23:09 geekosaur joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!