01:04 FROGGS_ joined
sergot hi o/ 06:14
06:59 itz_ joined 07:07 zakharyas joined 07:40 brrt joined 07:51 ggoebel1111112 joined 08:12 kjs_ joined
timotimo ohai sergot 08:29
(i'm a bit late to the party, but enh.)
brrt \o timotimo 08:31
what was your issue yesterday again?
nwc10 brrt: have you built with ASAN recently? 08:32
brrt .. no don't think i have
why? very broken?
timotimo brrt: i'm having difficulty matching up the output of the profiler against the jit log 08:33
brrt uhm... why is that timotimo?
nwc10 t/spec/S32-list/uniq.rakudo.moar t/spec/integration/99problems-51-to-60.t and sometimes t/spec/integration/99problems-11-to-20.t are failing
timotimo as in: i see routines in the routines tab that have ~100% spesh, but i see no bails whatsoever in the jit log
brrt jit log is lying
timotimo yeah
nwc10 for the same thing, and jnthn is stuck on why
timotimo i think it's because of inlining
brrt hmm
darn
i don't really have time to look at that right now 08:34
sorry :-(
timotimo that's okay
nwc10 (git submodules)-- # not transparent
brrt yeah submodules are problematic 08:35
nwc10: happen to know when they started breaking? 08:38
nwc10 not offhand, accurately 08:39
timotimo brrt: do you have hints where i should look next to advance moarvm and/or perl6? 08:40
nwc10 brrt: the IRC logs here remember better than me, and it's not conclusive: irclog.perlgeek.de/moarvm/2014-08-29
it seemed to be a rakudo change
odc hi, I just installed the latest rakudo star and i'd like to do some async stuff but I can't find find IO::Socket::Async anywhere 08:42
and google doesn't really help 08:43
FROGGS m: say IO::Socket::Async
camelia rakudo-moar a640aa: OUTPUT«(IO::Socket::Async)␤»
FROGGS odc: you do not need to import it, it is built in 08:44
into rakudo
odc err, but in the repl, when i type use IO::Socket::Async it says it cannot find it 08:45
FROGGS odc: do not 'use' it
like you do not need to 'use' Int or Str 08:46
it is there
odc oh
i see
thx FROGGS
FROGGS you're welcome
09:19 kjs_ joined 09:54 itz joined
brrt afk for lunch 10:02
10:02 brrt left 10:04 itz_ joined
jnthn evening, #moarvm 11:55
nwc10 heresy! 11:57
lizmat heresy? 11:59
jnthn China! :P
lizmat ah, that :-)
jnthn Also, I read the IRC logs in the morning in the taxi to work, which kinda counts as "on IRC" :P
carlin evening jnthn 12:00
jnthn Apart from this morning I forgot to grab 'em before setting off 'cus I got up too late. D'oh :)
carlin (in one minute it will be morning)
jnthn Still several hours to go until morning here :) 12:03
Which is good 'cus I didn't dinner yet. :)
12:07 brrt joined
nwc10 you've done this before haven't you? Forget to eat 12:07
jnthn nwc10: This ain't forgetting, just that the taxi driver took an insane route this evening, on top of leaving a bit later form $dayjob anyway... 12:20
Going out soon :) 12:21
nwc10 have fun, but not "fun"
jnthn Will try :) 12:25
bbl &
diakopter typical taxi driver scam 12:45
14:29 brrt joined
jnthn diakopter: No, I think this one was genuinely incompetent. There are plenty of scammer ones, though. :) 14:36
carlin: It feels a little odd putting the malloc wrappers in src/gc/allocation.* when they're actually nothing to do with GC-able allocation at all... 15:17
Also, while the #define MVM_free free is expedient, another static function would be more consistent (and easier to hack for those who want to instrument allocation for some reason) 15:18
Other than that I'm in favor, I think.
brrt nods that adding a real function for MVM_free is good for instrumentation (breakpoitns) 15:29
breakpoints
carlin should I make a new file, src/core/alloc.{c,h} perhaps? 15:42
jnthn carlin: Thta works fo rme. 15:43
wow. so typing... 15:44
carlin ok, I'll get onto that in a few hours 15:45
jnthn Cool. I'll take a look at the PR that results tomorrow and all being well can merge it :)
timotimo jnthn: do you know of a somewhat-LHF i could try to tackle in the coming days? 15:47
jnthn timotimo: If you fancy diving into the profiler code, it appears to me that the number of deopts reported on the front page doesn't equal the number seen on the OSR/Deopt page in all cases. 15:51
timotimo: I don't know which one is correct. I do know they can't *both* be correct. :)
timotimo: For a second LHF-ish that involves the GC but is maybe not terrifying but should be tried in a branch first: when I put in the profiler I started tracking the number of bytes promoted. 15:52
timotimo: Task: try to use that data in order to decide how often to do full gen2 collects as opposed to nursery ones. 15:53
timotimo ah
should the code also try to increase/decrease the nursery size in response perhaps?
jnthn timotimo: Today it's a fixed count. If you take a bunch of perl6-bench profiles that allocate a lot of short-lived things, you'll noticed we promote jack all, but still do a full collect every 25 runs, which is wasteful.
No, I don't think so. Not yet at least. 15:54
timotimo OK
jnthn Just play wiht having a megabytes promoted threshold for gen2 rather than a number of nursery collects threshold.
timotimo do you have an intuition what a good target value for "promoted bytes until a full collection makes sense" is?
jnthn I'd run a few benchmarks on promote-heavy things 15:55
timotimo all right
jnthn --profile-compile is pretty good for that
JSON parsing --profile is also good for that
It seems to me we often promote around 1MB per collect in those caess
Meaning that a 25MB threshold would be about equivalent to what we do today.
nine: FYI, I have an increasingly concrete design for DESTROY stuff on Moar forming in my head, which will get the thread affinity you need for the Perl 5 stuff. I'll likely work on a first-cut impl something within the next week. 15:57
*some time
timotimo hm, did i merge the string allocation pressure thing for the GC back then?
jnthn Don't think so.
Not sure we've missed it terribly since the strings refactor made us a lot less copy-heavy though. 15:58
timotimo right
jnthn For extra credit, you may want to look at the libuv API; there is a function that tells how much virtual memory the host system has.
timotimo that seems like it wouldn't require much code; the other LHF you mentioned doesn't seem that enticing to me at the moment :S 15:59
jnthn We might like to set the gen2 collect threshold based on that...
timotimo i've been dreaming about writing an analysis tool thet allows you to inspect the SSA form used in spesh in much more detail and with helpful tools like highlighting versions of the same register you're hovering or something like that
jnthn timotimo: Hm. It would turn a bunch of 25ms things into 5ms things in longer-running benchmarks, which could add up.
timotimo you mean reducing the number of gen2 collections, yes? 16:00
jnthn Yes
timotimo i'd imagine so 16:01
jnthn I mean, in benchmarks like for 1..lol-so-big-number { } we can reduce the numbber to zero with what I'm suggesting
'cus they promote nothing.
timotimo that'd be nice indeed
jnthn Uh, or the $i++ one for non-native $i, at least.
there's really no point doing a full collect when we promoted nothing.
timotimo how do you feel about reacting to "too much stuff promoted" by upping the nursery size?
jnthn I'm not sure. It's a *very* mixed blessing. 16:02
timotimo it could lead to "more time between collections gives more chance for things to die"
but also "longer nursery collections"
jnthn It may be less collects but (a) they'll be longer, and (b) make it bigger than it is now and you make hte active set of memory to consider larger than most folks CPUs have in on-chip cache.
timotimo oh 16:03
jnthn And once we miss beyond the L3 it's ugly.
timotimo right, caches are *way* important
jnthn I tuned it on my quite-cache-rich box and suspect we're already pushing our luck.
timotimo does windows give you a tool that shows you how many cache misses you get? 16:04
because time -v and perf somethingsomething gives you things like "percentage of branches mispredicted" and cache stuff and such
jnthn I'm also thinking of games like tadzik++ makes, which likely also promote very few objects, want fast collects often, and might also get to avoid gen2 collects if we change this...
timotimo mhm
jnthn those things on Windwos are typically exposed as performance counters 16:05
timotimo OK, good 16:06
jnthn Dunno if it's worth it or not to try and obtain such numbers to include in the profile. Will need platform specific code. Can always say N/A when we can't get 'em too... 16:07
OK, time for me to sleep
'night
timotimo gnite jnthn! 16:08
nine jnthn: awesome! Many thanks already! 17:10
17:48 Ven joined 18:15 kjs_ joined 18:38 nine joined 18:42 lizmat joined 18:46 PerlJam joined 18:51 brrt joined
brrt \o 18:51
[Coke] o/
brrt uhm, i have a bit of a theory wrt to the ASAN / valgrind dependent breakage 18:57
although no experimental evidence obviously 18:58
and maybe a quick disproving 19:00
19:17 Ven joined, vendethiel joined 19:32 kjs_ joined
timotimo hum. 19:36
i want to put a "collect all 'promoted_bytes' from the threads that have gc'd, sum them up and store them in the instance", but there's threads that are blocked and thus have their work stolen from them 19:39
brrt \o timotimo
timotimo how big is an AO_t? 19:42
like, is there a guarantee?
brrt uhm, yeah 19:43
i think an AO_t should be exactly one register wide
so 8 bytes on x64
dalek arVM: 21c5507 | (Bart Wiegmans)++ | src/jit/graph.c:
Add JIT logging of inlines
 19:44
brrt present for you!
timotimo sweet! 19:45
that means an AO_t is enough to store how many bytes we promoted since the last time we full-collected 19:47
then i can just MVM_add
brrt uh
i think so :-)
timotimo cool. 19:49
there's no good spot to reset the "bytes since last time we full-collected" to 0, because run_gc wants to inspect that for "do we want to do a full inspection?" before we kick stuff off and all the threads want to add stuff to it after they're done GCing 19:55
maybe i should just remember the value before we started and decrement the value by that amount afterwards
that should be robust
brrt i can't help you tonight, sorry :-) 20:00
brrt afk
20:00 brrt left
timotimo oke :) 20:00
nine Do I understand this correctly that MoarVM's GC moves objects in memory? What happens to NativeCall callbacks? Those pointers are somewhat out of reach for MoarVM. 20:02
timotimo i think callbacks are referred to with their CUID and stuff 20:08
the callback_data is malloc'd, so it doesn't move 20:09
nine ah, good to know, thanks 20:16
PerlJam random questions: a) Is MoarVM written entirely in C? b) What standard is it written to?
I ask because github thinks there's some C++ code in there and there are some .c files that have comments that start with //
(I think C99 groks //-style comments, but C89 does not) 20:17
timotimo MSVC doesn't like // 20:18
nine MSVC doesn't support C99 and never will
timotimo so if we commit // comments, jnthn has to fix them before being able to compile it
PerlJam Well ... lines 32 and 33 of src/strings/utf8.c look to have //-style comments and have been around since June 2012. I don't know what to make of that. 20:21
timotimo weird. 20:22
PerlJam In any case, I was just reading the code and became curious. 20:23
timotimo: there are even some lines that you added back in Feb in src/math/bitintops.c that have //-style comments :) 20:25
(git blame)++ :) 20:26
er, src/math/bigintops.c even
timotimo d'oh! 20:27
in that case ... i guess it doesn't bother jnthn's msvc
okay, moar's gc now tries to only full-collect if enough stuff has been promoted
i've yet to test this with multi-threaded workloads.
i'll have a look if it does the spec tests without being sad 20:29
japhb_ Regarding CPU cache info, I found a number of sources for doing autodetection. There's an API for Windows, and for Linux you have (of course) a selection of methods: shelling out to lshw, parsing from the pseudo-files /proc/cpuinfo and/or /sys/devices/system/cpu/cpu0/cache/index*/*, or in C using sysconf: gist.github.com/jesstess/797876 20:33
japhb (Just in case someone wanted to explore the auto-tuning angle, the info is there -- no need to benchmark and detect steps in the performance curve, which was the old way to do it) 20:34
lizmat decommute& 20:36
timotimo well ... that's no good. 20:38
ah, just forgot an & 20:40
dalek arVM/dynamic_gen2_tuning: 9ebdb71 | (Timo Paulssen)++ | src/ (3 files):
dynamically tune full collection interval

based upon how many bytes have been promoted to the old generation since the last time we did a full collection.		 20:47
carlin jnthn: github.com/MoarVM/MoarVM/pull/134 21:06
21:41 kjs_ joined
timotimo brrt: it doesn't seem like any "Entering" lines end up in the jit log at all 22:26
also, it seems like is_entry is set to 1 on both calls, so it'd also say "Entering" when it should say "Leaving"
TimToady it seems to run the parser a bit faster 22:31
timotimo "the parser"? like stage parse of core setting fame? 22:33
TimToady yes 22:34
maybe from running gen2 moar often? 22:35
so keeps a smaller set?
I dunno
timotimo should be "less often", no?
TimToady depends on how much intermediate stuff, I suppose 22:36
timotimo we still can't get a --profile-compile from the core setting :P
does somebody have a machine with lots and lots of ram to spare handy?
TimToady oh, to try to compile with only nursery?
timotimo no, to get the --profile-compile to find out how it's doing :) 22:39
TimToady slightly faster on spectest, 987 -> 980 cpu sec
oic
timotimo it'll compile fine and then use LOTS of ram to build up the json that it wants to put into the profile html page app thingie
TimToady how much memory we talking about?
timotimo on my weak 12gb of ram laptop, it won't get far
can't really tell how far it gets, though 22:40
TimToady
.oO(12gb as weak, sigh)
timotimo so maybe 32 gigs? 64 gigs?
128 gigs? :P
the browser will likely take a whole bunch of ram to run the app :D
ideally, we'd build some code to prevent the call graph from becoming OMGhuge without ruining the numbers
TimToady well, way outa my league... 22:41
but wait five years or so... 22:42
they've got a Cray sitting down the street in the Computer History Museum...oh wait, my phone is more powerful than that... 22:43
timotimo :)) 22:44
TimToady we just need to convince one of our computers that an SSD is really its main memory...
22:44 cognome joined
TimToady wonders off to see if he can figure out why .base(3) has burk 22:45
22:51 lizmat joined
TimToady oh, base.t is the 1..57 bug agaain 23:03
maybe... 23:04
anyway, last line of Rational.base is:$s ~= (0..9,'A'..'Z')[@f].join; 23:05
m: 0.1.base(10)
camelia ( no output )
TimToady m: 0.1.base(10).say
camelia rakudo-moar 83a5f5: OUTPUT«0.ABCDEFGHIJKLMNOPQRSTUVWXYZ␤»
TimToady or something is making a range sequence there 23:06
oh, wait, no, that's probably just a parcel needs flattening, dur 23:07
TimToady slinks back to #perl6

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!