github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
nwc10 good *, #moarvm 08:45
MasterDuke why would gist.github.com/MasterDuke17/1a307...821fe36c89 cause a sigabort and `corrupted size vs. prev_size` when doing `raku -e ''`? 11:30
timotimo what does valgrind say? 11:31
MasterDuke bin is 15 (so one of the ones i've changed) and size is 272
jnthn I don't think you changed it to actually allocate the page memory?
Oh, you did
MasterDuke gist updated 11:32
jnthn oh, but
your alloc limit is probably bogus?
timotimo yeah the limit is far too long
you're allowing fsa to write far, far outside of its pages
MasterDuke oh? 11:33
should it be unchanged?
timotimo what exactly were you going for?
i may have to explain what alloc_limit means
MasterDuke starting with more pages for some bins (vs starting with bigger pages for some bins) 11:34
jnthn Well, the allocator's design assumes that you are either allocating from the free list or, failing that, the latest added page
timotimo the alloc limit is a memory address, not a total amount
jnthn and yes, what timotimo said
timotimo when the alloc limit address is reached, a new page gets created
jnthn I don't really see how this change helps much, though, because you're making the same number of mallocs
timotimo i think i had suggested making a single bigger page first rather than making loads of pages 11:35
jnthn Yes, that would probably both help more *and* fit better with the overall allocator design 11:36
MasterDuke yeah, but we won't have to go through alloc_slow_path a lot
i'm just trying to measure a bunch of different options
timotimo it's possible that setting up multiple empty pages will just get the allocator to skip all but the first page
jnthn Or the last one, but yeah.
MasterDuke jnthn: i have some measurements from implementing what timotimo++ suggested, thought i'd also try this way 11:37
but i guess it's not (easily) possible
jnthn No, it goes against the grain of the allocator design a bit too much 11:38
MasterDuke afk for a bit. gist.github.com/MasterDuke17/0cde6...89c817ef53 has measurements for compiling CORE.c, gist.github.com/MasterDuke17/bf4bd...d8e892e481 for `raku -e ''` 11:40
timotimo another thing we can do is serve all the bins from a single malloc (which then would probably become an mmap) 11:52
ah, this "just" runs when the first bin is requested 11:53
nine Unless we're doing hundreds of mallocs here, I'd be sursprised if we could get a measurable speedup 11:57
timotimo but we are 11:58
36 adding a page of size 17408 for bin = 15
37 adding a page of size 8192 for bin = 6
314 adding a page of size 7168 for bin = 5
this is running raku -e '' on master
39 adding a page for bin = 1 11:59
56 adding a page for bin = 8
404 adding a page for bin = 2
406 adding a page for bin = 5
and this is with MD's patch to have VMArray use FSA
MasterDuke and it's not just the mallocs, it's trying and failing to go through the fast path and having to go through the slow path 12:13
i measured ~600k fewer instructions for `raku -e ''`
timotimo 0.01s system for raku -e ''; we should strive for that number to get to 0, though i wouldn't expect us to reach a point where the 0.01s are the majority of time spent 12:38
jnthn digs back into the dispatcher design work 12:49
timotimo and a whole bunch of the sys calls have always been brk
interesting, we actually shrink a couple of times, too 12:51
a plot of the brk numbers shows that the reductions are pretty much inconsequential :D 12:55
that's with out the MD patch,t hough 12:57
MasterDuke do people have an opinion on just making the first page (much?) bigger vs making the first page bigger and/or making subsequent pages bigger also? 13:34
should it just statically be for the bins that we see commonly allocated in some representative code?
or maybe dynamically resize the page size as more (or fewer) are requested? 13:35
timotimo difficult to say without a crystal ball
MasterDuke hm, maybe if i first write a program that will tell if another program will ever halt... 13:36
timotimo you can record a "trace" of all allocations for all size classes and then simulate what different algorithms would decide, and then find how much wasted space you would be allocating with any given algo
MasterDuke so model on some sort of theoretical knapsack and see how efficiently i can put different sized items in? 13:38
these are both pretty easy problems, should only take a day or two
timotimo i'd not call it that, honestly 13:39
since things of the same size always go in their corresponding bins
the decisions would only be what sizes you'd put for the bins
MasterDuke yep, just joking around
timotimo like, one approach would allocate exactly the maximum amount of memory used throughout the whole program at the start and then never have to do anything again
lizmat I wonder if memory usage information couldn't be kept somewhere, to give hints to any allocation scheme during runtime 13:47
a bit like spesh, I guess, but then over several runs
just brainstorming, and no I'm not suggesting you should drink bleach 13:48
jnthn While it doesn't tend to be done cross-process, there's all kinds of presizing, pretenuring, and pretransitioning optimizations that are done in this kind of area (not in MoarVM yet, but they are known/described techniques) :) 14:02
MasterDuke yeah, don't think i'm quite up for that level of work just yet. just some new simple heuristics 14:03
lizmat and another Rakudo Weekly News hits the Net: rakudoweekly.blog/2020/04/27/2020-...g-cleanup/ 15:02
jnthn Goodnes, I always figured that deferral was slow, but if you compare callsame against self.Parentclass::meth (the latter is already well optimized thanks to a spesh plugin) then it's more than 65x slower! 15:05
lizmat++ # weekly 15:07
MasterDuke wow. but you think the redesign will end up with it closer to the same speed? 15:08
jnthn Dunno yet :) 15:11
I'm quite happy with the non-resume-related bits of the design by now
Still trying to find something I like for the resume case 15:12
MasterDuke resuming exceptions? 15:18
jnthn No
Resuming dispatches
The thing that's going on with callsame et al. 15:19
MasterDuke ah 15:22
jnthn Finally, I think I found something that'll work... 16:11
timotimo cool! 16:39
jnthn Here's my design notes so far, for the curious: gist.github.com/jnthn/e81634dec57a...b92c722959 17:18
jnthn Home time & 17:20
nine jnthn: ah, I can see how that will help with the dynamically discovered methods in Inline::Perl5::ClassHOW :) 17:30
vrurg Is it possible for moar to know which module is being deserialized? I'm trying to dig into github.com/MoarVM/MoarVM/issues/1276 17:31
Altai-man_ vrurg, hi. Are you available for a bit? 17:32
vrurg Altai-man_: I think, yes, if nothing happens 17:33
nine vrurg: yes, look at the MVMCompUnit's file_name
vrurg nine: thanks! 17:34
Altai-man_ vrurg, can you please say what do you think about the next release proposal (gist.github.com/Altai-man/433f3314...55533eb3)? If the solution suggested won't work for you, we can discuss it. 17:35
vrurg Altai-man_: gimme a few minutes. 17:36
timotimo i wonder if there needs to be any special consideration for slurpy arguments that can help us deal with extremely long "arguments" / argument lists when trying to slurpy a very long array 17:37
[Coke] what do the trunc_xx opcodes do? 17:43
nine Truncate e.g. int64 -> int16 17:46
The opposite of extend
jnthn: oh, I just realized that this will finally get us super fast multi dispatch on constants like multi foo(1) { }; multi foo(2) { } by way of nqp::dispguardobj 17:49
[Coke] nine: would it be correct to describe trunc/extend as casting? 18:20
[Coke] m: use nqp; dd trunc_i8 123456789 19:34
camelia 5===SORRY!5=== Error while compiling <tmp>
Undeclared routine:
trunc_i8 used at line 1. Did you mean 'truncate'?
[Coke] m: use nqp; dd nqp::trunc_i8 123456789
camelia ===SORRY!===
No registered operation handler for 'trunc_i8'
Geth MoarVM: MasterDuke17++ created pull request #1277:
Make the first page for some FSA bins bigger		 19:35
MasterDuke m: use nqp; dd nqp::trunc_i8(123456789)
camelia ===SORRY!===
No registered operation handler for 'trunc_i8'
MasterDuke the above PR is to give something concrete to discuss, i'm open to changes 19:36
[Coke] Wonder if the test is misreading the nqp source and those aren't really opcodes. 19:37
(or if they are QAST only) 19:41
MasterDuke github.com/MoarVM/MoarVM/blob/mast...erp.c#L255 19:42
nine Could be they're opcodes but not actually mapped to NQP ops 19:43
They are used by the QAST compiler's coercions: github.com/Raku/nqp/blob/master/sr...T.nqp#L529 19:44
[Coke] I'll see if I can get them working in a QAST example. 20:10
timotimo you probably need to set a :returns(int16) in a qast node or something like that 21:08
Geth MoarVM: MasterDuke17++ created pull request #1278:
Fix some compiler warnings		 22:59

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!