samcv well, as a pointer to some malloced grapheme iterator i suppose. so we don't have to refind our place every time. since perl 6 or nqp let's say it requests graphemes one after each other, then it colud be faster to save the iterator 00:10
MasterDuke i'm just spouting stuff off the top of my head here, but if MVMStrings have to be immutable, what about creating a small pool of grapheme iterators? maybe accessed by the MVMStringBody's cached_hash_code? 00:25
samcv well i'm going to work on getting it faster to seek to a position 00:55
got from 1.1s to 0.9314294s for 'abcd' x 1000000 using indexic
MasterDuke cool 00:58
samcv going to test on non-repeat strands though 00:59
nice. from 0.9s to 0.8s :) 01:00
10 equal length strands ( very long strands btw)
3,227,579 chars per strand
this is for indexic because it still uses get_grapheme_at_nocheck instead of retaining a grapheme iterator 01:01
MasterDuke are short strings/strands also faster? 01:05
samcv yes 01:07
i mean regardless for every grapheme in that 10 strand file, it had to find its place again
so that means moving 0-9 strands forward
so it would show similarly for short ones too
MasterDuke great 01:08
samcv trying to get rid of the loop so that it only goes once 01:12
the second while loop in MVM_string_gi_move_to
01:31 committable6 joined, quotable6 joined, nativecallable6 joined, coverable6 joined, bisectable6 joined, bloatable6 joined, greppable6 joined, releasable6 joined, evalable6 joined, unicodable6 joined 01:32 benchable6 joined, statisfiable6 joined, squashable6 joined 01:51 ilbot3 joined 05:21 domidumont joined, brrt joined 05:28 domidumont joined
brrt good * #moarvm 05:39
06:03 KDr2__ joined 06:10 brrt joined 06:41 leont joined 06:50 domidumont joined 07:38 TimToady joined 08:01 zakharyas joined
timotimo good brrt 08:21
oh, i'm a bit late
brrt ohai timotimo
i'm still awake :-)
08:43 domidumont joined 09:13 Skarsnik joined
samcv good * 09:35
well i successfully got rid of the loop in MVM_string_gi_move_to that moves within a strand (or repeated strand) 09:36
yay speedups
and reduced the number of operations inside the loop that locates the strand
github.com/MoarVM/MoarVM/compare/m...gi_move_to 09:40
and removed and changed some code, added comments. since it was kind of confusing initially 09:41
hmm i think there's a bug where if we aren't at the first position in specific strand, aka gi->pos != gi->start and it's a repetition, then it could skip too much 10:00
strand_graphs = (gi->end - gi->pos) * (1 + gi->repetitions)
this is what we have now. but i think it's wrong. strand_graphs is the number of graphs left in the strand and lets us know if we need to go to the next strand, in case we are trying to move further than there are spots in that strand 10:01
so gi->end - gi->pos should give the space left in the current strand. but if we have repetitions, let's say gi->start is 0 and gi->end is 10. we're at 5 and there's 2 reps 10:02
(10 - 5) * (1 + 2) = 15. but i think it should be gi->end - gi->pos + (gi->end - gi->start) * (gi->repetitions) 10:03
10:03 zakharyas joined
samcv if jnthn is here let me know what you think. i think i'm right but i could be wrong 10:03
Geth MoarVM: 5bf652edbb | (Jonathan Worthington)++ | src/spesh/log.c
Don't decont native refs when spesh logging

That will result in the value being boxed. This allocation could make a problem by causing a GC run to happen, but much more importantly it leads to rather misleading spesh stats that could in turn end up with selection of a bogus multi-dispatch candidate during optimization. This thus fixes the mis-specialization that has caused failures in the atomics spectests.		 10:06
jnthn samcv: Is this in move_to?
samcv yep
jnthn Wasn't the situation a while back that we only ever expected it to be called once?
samcv yeah
jnthn I suspect it's a leftover assumption from then 10:07
samcv KMP doesn't skip more than one forward so we are currently okay
okay as long as what i'm saying sounds right :)
but i got ignorecase index from 0.9s to 0.8s for a 10 strand string
jnthn That is, if we weren't going from the start then we'd never be in the middle of a repetition.
Nice :)
samcv 10 copes of tolstoy and repeated some times
and removed the second loop 10:08
simplified some
also github.com/MoarVM/MoarVM/compare/m...50dc0efR96
can you tell me about this?
in the original code, we check if gi->repetitions < remaining_reps
but if the reps is less than remaining_reps, i don't get why we would set remaining_reps to gi->repetitions. that sounds like we should never get in that circumstance 10:09
where there are'nt eneough reps left
mathematically and practically unless there were corruption or something
since strand_graphs should calculate there's enough space left 10:10
or strand_graphs or whatever
jnthn remaining isn't just within this strand 10:11
So it's just saying "if the place we're going is longer than the number of repetitions, then eat all of them"
(But no more) 10:12
samcv that should never happen though?
jnthn Thus making sure gi->repetitions doesn't go negative
Thus ensuring that if (gi->repetitions) { doesn't falsely pass
samcv because we would be on the strand after it already
i changed that to if (remaining) instead of if (gi->repetitions) 10:13
since that seemed more reliable to count on
jnthn ah, you're saying we'd have skipped it above?
samcv and it should always happen at the same time. at least it never happened running roast
yeah 10:14
jnthn Yeah, I see that now
samcv and i also reforatted it so we have no loop below :)
which is pretty nice
jnthn Yeah, looks like an aubndance of caution costing us some cycles :)
samcv yeah no problem :)
always better safe than sorry tbh
jnthn But yes, agree it can be simplified :)
samcv and comments!!
:)
nice i get full spectest pass with changes :) 10:15
dogbert17 trick question: is there a reason why MoarVM isn't compiled with -Wall ?
10:16 robertle joined
timotimo all? why stop there! let's dial it up to -Wextra! 10:16
samcv why wstop wthere? wpreface wall words with w
dogbert17 well reading the discussion above it spits out warnings like: 10:17
src/strings/iter.h:90:23: warning: ‘gi.start’ may be used uninitialized in this function [-Wmaybe-uninitialized]
MVMuint32 rep_graphs = gi->end - gi->start;
perhaps a false positive but there's quite a lot of stuff like that 10:18
brrt dogbert17: we have that voluntarily, it's called clang :-P 10:19
in seriousness
i'm not against having that as a configure option
dogbert17 meets a -Wall of resistance :) 10:22
Geth MoarVM: 6d627da28d | (Jonathan Worthington)++ | src/spesh/stats.c
Handle native refs better in spesh stats

Previously, we would always consider a type tuple involving one as being incomplete, since a NativeRef is a container but we don't (any more) log the type of its content (and when we did, we got it wrong by logging the boxing of the actually referenced type).
This allows for generating observed type specializations of things that get passed native references. It's not enough, however, to see invocations of those be fastinvoke'd or inlined (at least, it not in the multi-dispatch case).		 10:39
jnthn Hah, yay, seems I got us inlining atomic int loads 10:50
Presumably (though will check) that means atomic int cas also inlines 10:51
nwc10 but first lunch? 10:54
jnthn Well, nodelay/blocking spesh stressing
But yeah, it's about lunch time
Looking good, spectesting with those stress options while I go for lunch 10:57
11:34 domidumont joined
jnthn yup, looking good 12:01
Geth MoarVM: fc75bca3c9 | (Jonathan Worthington)++ | src/6model/reprs/MVMMultiCache.c
Support spesh multi resolution involving nativeref

This means that we can fastinvoke or inline candidates being passed a native reference. This makes, for example, the load of an atomic int possible to inline, but likely enables a lot of native inlining more generally.		 12:03
12:07 committable6 joined 12:12 Skarsnik_ joined
jnthn brrt: The EXPR JIT works at BB level at the moment. I'm wondering if we might not get a lot more bang for buck even with that restriction if we were to make a pass of fusing basic blocks that needn't be basic blocks thanks to the elimination of invokish things and other code... 12:12
brrt yes. 12:13
that would be pretty cool
nine So much for "no room for improvement" ;) 12:15
brrt but then, we also need 100x more templates :-P
hehe, it's more like, 'where on earth should we start'
re, invokish ops 12:16
jnthn nine: heh, there's tons of room for improvement :)
brrt repr-specific templates, i could imagine replacing an invokish implementation with a noninvokish one 12:17
e.g. for stringification and stuff
jnthn Hurrah, cas for integers is indeed being inlined
brrt so i think, a template should carry some flag that indicates it isin't invokish 12:18
which should overrride the per-op flag
we should porbably keep the per-op flag though
jnthn Aye 12:19
Many invokish ops are rewritten to non-invokish during specialization, though
brrt true 12:20
jnthn Hm, next curiosity: why is the == in `last if cas($value, $orig, $orig + $i) == $orig` being fastinvoked but not inlined... 12:23
ilmari huh, none of the routines in docs.perl6.org/type/atomicint appear to be indexed 12:25
jnthn wonders why 12:27
jnthn has no idea how indexing works, though
I figured it went on headings or some such
ilmari the operators are, though
jnthn Hm, even more odd 12:28
Open a docs ticket, mebbe 12:29
Somebody probably knows :)
ok, the == ends up with an assertparamcheck left behind 12:30
brrt and there was another thing with the multiple basic blocks… current expr trees only have a 'forward' CFG, and with multiple BBs, i need to handle backward ones as well, and that is mostly relevant for the live ranges holes finding code 12:31
jnthn ah, yes
Think I can see what's going on with the == thing
Hmm
Guess fixing this will make a lot of code mixing native/non-native integers happy...
ilmari ah, they're "=head1 Routines \n =head2 foo", instead of "=head1 Methods \n =head2 routine foo", which seems to be the standard 12:33
jnthn Hurrah, the == is inlined there now too 13:26
timotimo wowza, i might just have kicked a mid-life crisis out of the heapanalyzer's "parse snapshot" code 13:34
gc time went from 25% to 15% 13:35
suddenly 1000 OSRs more than before 13:36
Geth MoarVM: ab28683b2f | (Jonathan Worthington)++ | src/spesh/optimize.c
Set facts on nativeref deconts.

It would, at some point, be nice to have better code-gen for the decont/box itself. But for now, we can get a notable win from setting up the facts on the target of the decont. We know the type of the result and also that it will be concrete, which in turn allows for full elimination of parameter checks in the case infix:<==>(Int, Int) ... (5 more lines)		 13:38
jnthn With that, something like
loop {
my int $orig = $value;
last if cas($value, $orig, $orig + $i) == $orig;
}
Will be fully inlined
(the last, cas, and ==) 13:39
brrt jnthn++
timotimo i like the sound of this
i shall grab latest moarvm and see if it changes things in the heapanalyzer, too 13:40
nwc10 in the bigger picture, what does this buy us - less CPU burned doing concurrency? 13:41
timotimo 2x as much inlining, but also 2x as many just-interpreted frames 13:42
jnthn nwc10: Taken together, these help pretty much any code that passes around native references 13:43
By adding various missing pieces that block inlining (or just cheapening the call by reducing duplicate checks) 13:44
Code using the atomic ops on native ints gets a particular win though 13:45
'cus this is exactly what it does
14:04 zakharyas joined
timotimo now collectables are parsing faster than references (in this case there's 456_418 collectables and 1_761_280 references) 14:07
sadly, the code is full of nqp::shift_i and nqp::push_i
jnthn Hm, darn. So it seems that I can now have us re-compute the dominance tree before the second pass 14:11
timotimo ooooh 14:12
jnthn However, doing so causes the second pass to start descending into things it never did before. It then manages to mess things up enough to send some code into an infinite loop.
timotimo did you do the work necessary to put inlines in the "correct" spot?
jnthn No
timotimo OK, i see
jnthn That's a separate task
timotimo i still have a branch for that that i probably didn't push; it doesn't work properly, but it's a start
samcv jnthn, i confirmed it 14:20
the old code doesn't work
if you move the iterator after it's already been moved
in MVM_string_get_grapheme_at_nocheck i had it randomly move the iterator until it finally gets to the end destination, and i get it iterating too far 14:22
timotimo i find it annoying that weechat decided to make samcv, nwc10, lizmat, and brrt have the same color
samcv you can probably change it
to have 256 colors
timotimo i believe i did 14:23
there's a faq entry that has the magic incantation
timotimo found another big optimization opportunity 14:29
my uint64 @result .= new(1, 2, 3, 4, 5); is 2x slower than my uint64 @resulf = 1, 2, 3, 4, 5;
is 2x slower than my uint64 @result; nqp::push_i(..., 1); nqp::push_i(..., 2) ...
dogbert17 jnthn: looks as if one of your fixes has nailed RT #130855 14:31
synopsebot6 Link: rt.perl.org/rt3/Public/Bug/Display...?id=130855
jnthn ooh, interesting 14:33
dogbert17 dunno which one though
14:33 brrt joined
jnthn Wonder how far back we could be talking :) 14:34
ilmari bisect: sub foo () {$ = 42}; for ^2000000 { $ = foo } 14:35
bisectable6 ilmari, On both starting points (old=2015.12 new=4b02b8a) the exit code is 0 and the output is identical as well
ilmari, Output on both points: «»
ilmari it fails for me on This is Rakudo version 2017.06-198-g58900e7ba built on MoarVM version 2017.06-56-g161ec639 14:36
timotimo 33.92user 0.65system 0:13.36elapsed 258%CPU (0avgtext+0avgdata 848288maxresident)k 14:37
119.40user 0.83system 1:27.46elapsed 137%CPU (0avgtext+0avgdata 1188220maxresident)k
Zoffix bisect: old=58900e7ba,new=HEAD sub foo () {$ = 42}; for ^2000000 { $ = foo }
bisectable6 Zoffix, Bisecting by exit code (old=58900e7 new=4b02b8a). Old exit code: 1
Zoffix, bisect log: gist.github.com/1227bf575dca207b9f...40ad1e1cb2 14:38
Zoffix, (2017-08-11) github.com/rakudo/rakudo/commit/76...ed2e2fe011
ilmari "af24ab8 Can never inline frames declaring state vars." looks like a likely candidate 14:39
timotimo since it uses so much ram, it perhaps gets a performance penalty from swapping on my laptop 14:41
jnthn ilmari: ah, yes, that'd do it :) 14:42
timotimo flips the order of heapsnapshots and shared data around in the heap snapshot format 14:49
Geth MoarVM/heapsnapshot_binary_format: 6063981424 | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
sprinkle comments and empty lines for readabiltiy
 14:55
MoarVM/heapsnapshot_binary_format: acbbde09b8 | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
move snapshots to beginning of file

this way we can stream out the snapshots every time we GC and then free their memory, then later just finish it up with the string heap, types, and static frames.
timotimo that was easy, wow. 14:56
15:04 travis-ci joined
travis-ci MoarVM build errored. Jonathan Worthington 'Support spesh multi resolution involving nativeref 15:04
travis-ci.org/MoarVM/MoarVM/builds/272027364 github.com/MoarVM/MoarVM/compare/6...75bca3c9a2
15:04 travis-ci left
timotimo timed out on one of the mac builds ^ 15:05
timotimo notices that all of this was just because he wanted to see if the describe functions for the new spesh structures are correct 15:09
isn't yak shaving fun
Zoffix chuckles that google's result for "wiki Yak Shaving" leads to a page that says "Wikipedia does not have an article with this exact name. Start the Yak shaving article" 15:14
jnthn :P
15:14 zakharyas joined
japhb Zoffix: :-D 15:23
jnthn Of course, the place where the set elimination blows up would be in a frame with 14 inlines :P 15:28
timotimo my advice: just throw it out completely 15:36
have you tried the spesh step-by-step tracing gdb script i built, jnthn?
jnthn Not yet 15:40
15:47 lizmat joined
jnthn So the interesting thing about this case is that at the end of an inline we have 15:57
sp_fastinvoke_o r204(3), r205(2), liti16(2) 15:58
set r206(2), r204(3)
[Annotation: FH End (12)]
set r123(10), r206(2)
[Annotation: Inline End (5)]
goto BB(1121)
Being rewritten into
sp_fastinvoke_o r123(10), r205(2), liti16(2)
[Annotation: Inline End (5)]
goto BB(1121)
(yes, the FH end moves up above the fastinvoke, it's fine)
Where r123(10) is outside of the inline's register set
(It's the caller register) 15:59
timotimo right. at first glance that looks like a decent optimization (unless of course it doesn't update the facts properly?)
jnthn Don't think facts come in to it. This is the only one of the 3 set elimination transforms that actually *doesn't* screw the facts
The other two do 16:00
timotimo OK
jnthn Which will have to be fixed later
timotimo how does it make things explode?
jnthn I don't know. My first guess is deopt
timotimo one of the registers that are being skipped were used post-deopt at that point?
jnthn The end result is an infinite loop of some sort
timotimo like, we deopted into the one that would return a value from a register that was no longer being set 16:01
jnthn ah, yeah 16:05
For the topmost uninlinee we do this:
MVMuint16 orig_reg = (MVMuint16)(f->return_value - f->work);
MVMuint16 ret_reg = orig_reg - cand->inlines[i].locals_start;
uf->return_value = uf->work + ret_reg;
timotimo we grab the contents of the register that used to get used for the return in question? 16:06
jnthn Note the substraction
orig_reg - cand->inlines[i].locals_start 16:07
That'll become *negative*
And point to...heck knows where
timotimo oooh, ouch
because we changed the register in the op
d'ouch
jnthn Right
jnthn ponders the best way to deal with this 16:08
timotimo i can't believe we didn't have a string constant for "filename" or "path" or something yet 16:12
jnthn I guess any invokish is implicated 16:16
Oh, maybe not
Hmm
Yeah, actually it perhaps is.
I guess the boring answer is to just not allow a rewrite to the outside of the current inline 16:17
timotimo hm, right 16:18
jnthn Though we don't immediately have the info available to do that 16:19
(We don't store *which* inline a basic block maps to in a convenient way)
Ah well, think I might leave it for tomorrow, getting a tad tired now 16:20
timotimo sure thing
good work so far!
we can just toss the rewriting opt for now ;)
jnthn Well, we need 'em in the end, so... 16:21
The reason I set off trying this was that I wanted to move the exception handler -> goto opt to be post-inline
Spotting that it would apply to the cas loop but also to other things
But since we didn't have the inlines in the dominator tree, well, we'd not get it 16:23
Geth MoarVM: 9a5987f030 | (Jonathan Worthington)++ | src/spesh/optimize.c
A little refactoring of second optimization pass

No functional changes, just preparing it for doing a bit more, and breaking the code into easier to reason about pieces.		 16:24
MoarVM: 73bfaf414f | (Jonathan Worthington)++ | src/spesh/optimize.c
Remove fact copy we most likely don't now need

Since propagating information during the optimize phase is so critical to getting more inlines and so forth, we'll already have done this by the second phase.
MoarVM: 3271cd8d51 | (Jonathan Worthington)++ | src/spesh/optimize.c
Re-flow and better explain set elimination code

Done as part of understanding its shortcomings. Also comment on the places where it needs to be improved to not result in a busted spesh graph.
jnthn Those are some updates I've done along the way :)
Will hold the other bits in reserve until tomorrow, when I figure out how to solve the inline issue. 16:25
lizmat jnthn: does MoarVM / libuv have some knowledge about currently open files?
jnthn No 16:26
lizmat ok
jnthn I mean, technically MoarVM can walk the entire heap
And find every file handle
Which is how the heap profiler and GC do things
lizmat well, I was more thinking along the line of an array indexed on native-descriptor 16:27
jnthn But there's no other way of tracking things
No, there isn't that
lizmat a .close would clear the entry, on exit it would close all the entries in the list of open filees
it still wouldn't close on scope exit like in Perl 5, but at least it would ensure buffered output would not be lost 16:28
timotimo the next step for the heap snapshot format would be to have pieces of strins/types/staticframes before or after every single snapshot, but only containing the difference. that way, even a ctrl-c'd program would give usable & useful data
jnthn It's just hiding a problem though, which can eventually manifest itself in too many open files.
lizmat jnthn: which could be important in chasing bugs causing programs to exit
jnthn: true 16:29
jnthn If you're writing such a log, just open it with :!buffer
geekosaur yeh, there's a reason C makes sure stdio filehandles are recorded for atexit() to flush. I'm thinking something similar, at least for buffered handles, would be smart
jnthn We already are doing it for stdout/stderr
geekosaur if you are writing a log from a program that crashes, you lose diagnostic information
and disabling buffering makes that logging expensive 16:30
jnthn True
lizmat jnthn: how is it done for stdout/stderr ?
jnthn Because those handles are created by the VM and hang off ->instance
We can of course keep an array of all open files, and remove stuff fromit 16:31
So long as you accept that the GC will then *never* close a file for you if you forget, so every opened file hangs around until program exit
geekosaur if it isreally hard /expensive to do for all handles then maybe a new parameter to add to an autoflush list would be an idea
jnthn I could live with that trade-off I guess
In that it will also make resource-leaky programs fall in a heap faster 16:32
geekosaur but really I think the 'hangs around' issue will likely force action on this at some point
jnthn geekosaur: 'hangs around' issue?
16:32 travis-ci joined
travis-ci MoarVM build passed. Jonathan Worthington 'Set facts on nativeref deconts. 16:32
travis-ci.org/MoarVM/MoarVM/builds/272058760 github.com/MoarVM/MoarVM/compare/f...28683b2f2f
16:32 travis-ci left
geekosaur doesn't get closed automatically when unreachable 16:33
jnthn geekosaur: Ah, you're saying that if we never do that, then people will be forced to fix their stuff?
geekosaur: Or that this would count as a bad behavior?
geekosaur because if you can't currently do it except by explicit close then things kinda get nasty. I don't think we quite have Haskell's issue there (basically Haskell *must* close stuff when it becomes unreachable, because laziness means you otherwise have no rel control over how many open handles you have)
but if making an expression lazy can cause this then yes, I foresee problems in the future 16:34
jnthn I don't really see, short of the huge yak shave involving implementing weak references, how we can both have close on unreachable *and* keep a list of open things to close at exit.
geekosaur of course we don't have Haskell's style of lazy I/O either (unsafeInterleaveIO is convenient but has a ton of traps associated with it) 16:35
lizmat my view:
1. you should have something like a my $handle will leave { .close } 16:36
geekosaur basically I am envisioning a situation where some object has a handle embedded deep within it and eventually becomes unreachable. of course that's begging for EMFILE issues anyway, but sometimes it's the best option for doing some things 16:37
lizmat 2. if you don't, you will eat up resources wrt to number of open handles
3. if you exit, MoarVM should properly close all the open file handles before exitin
jnthn Well, there's the two of you wanting the opposite solution :P 16:38
geekosaur jnthn, actually I am thinking handles could go in a separate gc "bucket" so that at exit only that bucket could be gcd
Zoffix 's most common IO doesn't touch handles at all...
geekosaur and only gcd for the handles not for general objects, but that may be special casing things a bit too much 16:39
jnthn It is.
The GC only has two spaces, nursery and gen2 16:40
lizmat in my opinion, the damage from losing buffered data is much larger than the damage of too many open handles
Zoffix Make that bucket more generic, so you could have Perl's equivalent of DESTROY that's guaranteed to run on exit
class Foo is Collectable {... }
Zoffix is talking from complete ignorance of GC stuff, mind you :)
jnthn lizmat: Note that you can implement what you're suggesting in Rakudo 16:41
lizmat jnthn: yes, and I'm tempted :-)
which would solve the issue also for other backends, I guess :-) 16:42
jnthn Indeed.
And means it's more clear for others to see what happens
Also
If at some point we do get weak references, then we can have our cake and eat it
(By making the open handles list be a list of weakrefs, so they won't keep handles alive if nothing else would)
If we do it at Perl 6 level we also get the chance to make the feature opt-in at Perl 6 level should we need to in the future. 16:43
As well as making it easier to build features like "store a stack trace with each open if an env var is set, and at exit show me all the handles I leaked" 16:44
lizmat ok I'll put that on my list 16:46
jnthn Alright
lizmat meanwhile, I need to decommute&
geekosaur mrrr. and now I am thinking 'make that resources instead of just handles' :p
jnthn fwiw, weak refs are possible, just...need notable effort
geekosaur (ResourceT in Haskell)
jnthn Step 1 being going to re-read the appropriate chapter of the GC handbook so I do it in a sensible way rather than re-inventing a poor one :) 16:47
When I may find it's not quite as bad as I fear...
But even then we need to figure out how we'll expose it at Perl 6 level, and if we need things like a callback when the thing gets collected 16:48
(Otherwise, you never reclaim your array slot)
And all that fun
Probably in the medium term, it's what we want though 16:49
[Coke] jnthn: Looks like 1.0.2 is the latest version in macports of openssl, btw. 16:56
jnthn [Coke]: Yeah, the ALPN love is going to be slowly shared, it seems... 16:57
Skarsnik hm I replaced github.com/MoarVM/MoarVM/blob/mast...all.c#L810 with cptr = (void*)((uintptr_t)storage + (uintptr_t)repr_data->struct_offsets[i]); 17:02
and it make efence happier
[Coke] oops. wrong window, sorry 17:05
jnthn wanders home 17:08
17:26 zakharyas joined
timotimo fun, when a heap snapshot file has been truncated by a ctrl-c'd process, it won't have the index at the end 18:22
unless i write out a full index after every bit, that would work, because there'd always be an index at the end 18:27
geekosaur and here you see why I mentioned ResourceT earlier :p 18:29
18:48 dogbert2 joined 19:31 zakharyas joined 19:46 lizmat joined 19:58 ggoebel joined 20:01 greppable6 joined
timotimo this is C-level :) 20:08
Skarsnik it's me or github.com/MoarVM/MoarVM/blob/mast...all.c#L746 and github.com/MoarVM/MoarVM/blob/mast...all.c#L795 could probably be merged? 20:55
geekosaur are the masks etc. all guaranteed to be the same? 20:56
Skarsnik I will look at that tommorow x) 20:57
Geth MoarVM/heapsnapshot_binary_format: a8faddedd7 | (Timo Paulssen)++ | 5 files
WIP on streaming snapshots to file on every gc
 21:28
timotimo (just for transferring to my desktop) 21:29
MasterDuke timotimo: i tested it a tiny bit yesterday. took 25s to `summary` a snapshot of `perl6 -e ''` 21:31
yoleaux 06:19Z <AlexDaniel> MasterDuke: maybe you have some idea. committable.t and benchable.t hang on the first “Did you mean …” test
06:23Z <AlexDaniel> MasterDuke: sorry, bisectable.t and benchable.t. So github.com/perl6/whateverable/blob...ble.t#L165 and github.com/perl6/whateverable/blob...ble.t#L142
06:23Z <AlexDaniel> MasterDuke: it may be associated with Sift4 module, which is problematic by itself… for example, it's still leaking memory.
MasterDuke which i thought was a long time, so i timed the old heap analyzer, 100s for the same thing
i just usually go do something else and didn't realize exactly how long it did take before! 21:32
timotimo MasterDuke: how big is the file, how good is the multi-core utilization? 21:33
also, which commit did you test with? i made things even faster still 21:34
MasterDuke 22mb, don't remember exactly, but think i was seeing ~315% cpu in top
timotimo that's not so bad
MasterDuke 16d9c2220187018410ae6a482398920b170e0a07 i think 21:35
timotimo you should be able to check out the last commit before head and try again
MasterDuke cool, i'll do that sometime this evening 21:36
timotimo nice 21:38
21:39 AlexDaniel joined
timotimo the next changes i'm making aren't about making things faster, i'm afraid 21:39
they will make running a program with the heap snapshot profiler active a bit cheaper on memory, though
MasterDuke that's good too 21:40
timotimo i didn't actually measure yet, though 21:51
21:55 travis-ci joined
travis-ci MoarVM build passed. Jonathan Worthington 'Re-flow and better explain set elimination code 21:55
travis-ci.org/MoarVM/MoarVM/builds/272129206 github.com/MoarVM/MoarVM/compare/a...71cd8d5158
21:55 travis-ci left
timotimo wow, that took ages 21:55
timotimo remembers we have moarvm.github.io/coverage 21:56
samcv is this right? an empty string is found after a string, as in 22:12
m: use nqp; nqp::index('hi', '', 2).say
camelia 2?
samcv m: use nqp; nqp::index('hi', 'i', 0).say 22:13
camelia 1?
samcv that's what perl 5 does at least so i guess it must be right 22:14
MasterDuke m: dd 'a'.split('') 22:15
camelia ("", "a", "").Seq?
samcv ah 22:16
MasterDuke i think there was some chat about this in #perl6 recently 22:17
maybe around here irclog.perlgeek.de/perl6/2017-08-29#i_15086059 22:18
Geth MoarVM: samcv++ created pull request #675:
Optimize MVM_string_gi_move_to, fix if not starting at 0		 22:28
timotimo getting closer to a working version of the "streaming" heap profiler 22:33
just threw heap snapshot data dumping out of nqp
(and moved generating the filename up front and all that) 22:35
m( i called it path in one and filename in the other 22:38
samcv using grapheme iterator cached for indexic i go from 1.6s to 0.78 seconds for a 10 strand string 22:44
nice
timotimo neat 22:45
samcv bringing it in line with the speed for a flat string 22:47
MasterDuke ! nice 22:52
timotimo another rebase coming up 23:05
Geth MoarVM/heapsnapshot_binary_format: 9 commits pushed by (Timo Paulssen)++
timotimo just tested with multiple snapshots in one file 23:19
and i *think* i can insert garbage between snapshots so making heap snapshots "recoverable" on crash should be backwards compatible with earlier heapanalyzers 23:22
hmm, then the size entries for snapshots will be bigger, that'll be weird, but it'd still work
i might want to introduce an extra field for that before i go on. 23:23
but first i'll have some sleep 23:24
23:40 eater joined
samcv i think i've decided how i'm going to do KMP with really long needles. if it's too long to fit in the pattern, i'll only processs ~8000 needle graphemes, and search for that. if it finds it, i'll do eqat for the remainder of the needle. if it's a match we return the position. if it was false we continue searching 23:45
still not sure if 8000 is the correct length for the jump table or what the best max length is. but that seems sane enough and not too big 23:46
can i add an opcode that will tell you what representation a string is in 23:49
or do we have too many ops already :P 23:51
but it could be useful for people who use moarvm because i don't think they have a way to find out what rep the string is in.

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!