03:57 vendethiel joined 07:56 rurban joined 08:19 Ven joined
|dalek||arVM: 15908a2 | jnthn++ | src/gc/roots.c:
Mark the cached backend config hash.
|nwc10||jnthn: much rakduo SEGVing on linux. at this line of moar.c:||08:46|
|MVM_interp_run(tc, &toplevel_initial_invoke, start_frame);|
|seems to be that the address of start_frame has the lowest bit set|
|jnthn||nwc10: Did you have 15908a2?||08:47|
08:47 Ven joined
|nwc10||I think I was on 15908a2^||08:48|
|jnthn||OK, but...weird. I've a build on Linux here too and all's well with latest of everything|
|nwc10||I'm trying to figure out what caused it||08:49|
|jnthn||m: say 9611866 - 9552454||09:36|
|camelia||rakudo-moar f3ecf8: OUTPUT«59412␤»|
|jnthn||Another 60KB off CORE.setting||09:41|
|That's 60KB less *bytecode* to verify and interpret||09:42|
|nwc10||with the cached backend config?||09:43|
|That was a smaller win (about 1.5 million less CPU instructions)|
|Well, a different kind of win, rather.|
|nwc10||spectest seems to be happy with Moar at 15908a2||09:45|
|(Rather than 15908a2^)|
|jnthn||Oh, I get a nice CPU win too||09:47|
|m: say 541034601 / 552393040||09:48|
|camelia||rakudo-moar f3ecf8: OUTPUT«0.9794377587␤»|
|jnthn||Another 2% off startup. I'll take that.|
|dalek||arVM: 0acd74b | jnthn++ | src/ (6 files):
Dual-purpose comp unit string heap as SC one.
This means we don't have to construct an array of strings in the order the deserialization blob wants them, and can instead just point the deserialization at the string heap of the compilation unit. When the NQP change to take advantage of this is applied, we knock 60KB off the size of CORE.setting, and save 2% off startup time.
|jnthn||m: say 541034601 / 644090520||09:54|
|camelia||rakudo-moar f3ecf8: OUTPUT«0.8399977708␤»|
|jnthn||Now running only 84% of the instructions at startup we used to compare to yesterday.|
|dalek||arVM: 5179453 | jnthn++ | src/core/bytecode.c:
Re-order bytecode read struct to be smaller.
|arVM: 48171d9 | jnthn++ | src/core/bytecode.c:
Don't repeatedly calculate limit in bytecode read.
Also mark it MVM_INLINE_STATIC and remove a bogus NULL check.
|jnthn||m: say 540112560 / 541034601 # a tiny but easy win||10:10|
|camelia||rakudo-moar f3ecf8: OUTPUT«0.9982957818␤»|
10:18 Ven joined 11:43 Ven joined 12:14 Ven joined 12:20 rurban joined
|timotimo||cool, i like these changes||12:57|
13:17 vendethiel joined 14:02 vendethiel joined 14:39 mj41 joined 14:56 zakharyas joined
|dalek||arVM: 6b7a61d | jnthn++ | 3rdparty/uthash.h:
Toss some bits of ut_hash we don't use.
As part of preparations for more extensively modifying it.
|arVM: cf4ef03 | jnthn++ | 3rdparty/uthash.h:
Toss hash bloom test bits we don't use.
|if we heavily modify it, maybe we'll end up having MVMString objects that actually store latin1 if they can|
|er, MVMString isn't an "object"|
|but you know what i me|
|jnthn||It is an object as far as the GC cares :P||15:28|
|But yeah, that's kinda where I'm going :)|
|memory usage could go down a whole lot from this|
|on the weekly, someone commented having NFG strings for everything could lead to problems because it balloons up the memory usage of a single character even if it could be stored in latin1||15:31|
|like if you analyze log files or something and the log file happens to be a gig in size|
|jnthn||If you're reading it all into memory you may already have issues ;)||15:32|
|But yeah :)|
|jnthn||I think if you explicitly decode the thing as latin-1 then we can probably get it stored byte size.||15:34|
|since we have ropes, we could also decode until we find something that latin1 can't handle and switch over and have a two-part string|
|except if the latin-1-codable prefix is too short|
|dalek||arVM: 315aac7 | jnthn++ | 3rdparty/uthash.h:
Toss unused hash signature.
We don't make any use of this, so it's just 32 wasted bits.
|jnthn||timotimo: Yeah, I'd pondered that too :)|
|timotimo||maybe i ought to freshen up the heap analyzer plugin to gdb and have a look at what's on the heap these days||15:37|
|last time i looked we had multiple copies of the same MVMString in the nursery for a big amount of strings||15:38|
|i wonder where that came from and if it's still there|
15:45 Ven joined
|dalek||arVM: 77e3fbc | jnthn++ | Configure.pl:
Optimize at level 3 by default.
We've been at 2 for a good while without issues; let's bump it a tad further. Doesn't make a big difference, but every little helps.
|arVM: 42d7dd8 | jnthn++ | 3rdparty/uthash.h:
Toss even more hash bits we don't use.
16:03 Ven joined
|timotimo||i've been on -O3 for a long time now||16:14|
|(and i even -march=native)|
|jnthn||This is the best error I've got out of a C compiler in ages!||16:26|
|C2064: term does not evaluate to a function taking -1000 arguments|
|jnthn is further hacking up the code once known as UT hash||16:32|
|I'm currently working on getting rid of the linked list for the order stuff was added in.|
|Which'll save us 16 bytes (on 32-bit machines) per item put into a hash.|
|I *think* it should be do-able without a total re-write of the thing. I hope. :)||16:33|
16:57 rurban joined
|nwc10||sort of a too-late question now - you started on the version we had, not the most recent (and now abandoned) version upstream?||17:25|
|jnthn||nwc10: No 'cus we already have some more minor modifications, though looking at the change log I think pretty much all that we'd have missed was in the code I tossed anyway||17:27|
|Seems I've managed to re-work the HASH_ITER macro to go through the buckets rather than needing the ordered linked list.||17:29|
|jnthn spectests that piece|
|I think there's another iteration path I'll need to change.||17:30|
|And then can see if tossing the addition order linked list is as easy as I hpe.|
|...yeah, apparently it's dinner time too :)|
18:10 vendethiel joined 18:44 vendethiel joined
|dalek||arVM: 7c1aaf7 | jnthn++ | / (9 files):
Re-implement HASH_ITER in terms of buckets.
This is the first step in removing the use of the double-linked list that maintains insertion order, which from a NQP/Perl 6 perspective is simply time/memory overhead that we don't need.
|lizmat||which means we finally get randomly ordered hashes ?||18:52|
|jnthn||lizmat: I don't think MVMIter uses the macro, so that's next in line||19:03|
|But yeah, we'll lose insertion order|
|jnthn||"randomly" is maybe a bit strong :)|
|lizmat||wonder how much this will break in the ecosystem :-)|
|jnthn||I hope not much given that JVM hasn't given the ordering||19:04|
|lizmat||weill, it'll allow some .pick(*) in some .perl's, to be removed||19:06|
|timotimo||keys will be placed pseudorandomly depending on their values as well as insertion order||19:13|
|we'll have to do a bit of extra work to make the order different on every execution of the same script||19:16|
19:17 Ven joined
|lizmat||I don't think that's necessary just yet||19:17|
|jnthn||timotimo: We'll likely have to do that to avoid security issues|
|lizmat||well, before 6.0 sure :)||19:18|
|jnthn||Thankfully there's some good knowledge on these issues in the Perl community|
|lizmat||true, but it's also a source of discussion, and sometimes feuds :-( let's not go there :-)||19:19|
|jnthn||It's fine, I only have to listen to the people who are right. :P||19:25|
19:25 zakharyas joined
|lizmat||just like picking the right standard :-)||19:25|
|hmmm... I just realized this may break typed hashes||19:26|
|as it uses 2 hashes: one for the keys and one for the values|
|jnthn||Should be fine, I think...||19:30|
19:31 Ven joined
|lizmat||yeah, HashIter is doing a lookup||19:35|
|jnthn: do you have any idea why it was done that way, rather than having a single hash with Pairs (original object => value) ??||19:36|
19:36 Ven joined
|lizmat||generation of .pair/.kv and friends would be comparable, without the need of keeping a whole 2nd hash with identical keys||19:37|
|fwiw, that's how I implemented bags/mixes|
|jnthn||lizmat: Because so far we've only got (at the lower level) support for hashes with string keys||19:39|
|lizmat||yes, I get that|
|but why not the approach of StringKey => ( Key => Value )||19:40|
|instead of (StringKey => Key), (StringKey => Value)|
|internally, I mean|
19:41 Ven joined
|lizmat||the natural form of a Hash is a list of pairs, which is easily generated from the former setup :-)||19:41|
|jnthn||Ah, I see your point|
|Not sure; what you're suggesting could be better|
|lizmat||and is proven, as bags/mixes are implemented that way :-)||19:42|
|nwc10||jnthn: ASAN! paste.scsys.co.uk/475870||19:43|
19:46 vendethiel joined
19:56 cygx joined
|cygx||jnthn: while you're working in the general vincinity - I just got a ===SORRY!=== Missing serialize REPR function for REPR VMIter||19:56|
|jnthn||cygx: That's probably quite orthogonal.||19:57|
|cygx||jnthn: just saw an MVMIter in the logs|
|in related news: hello world from the Tiny C Compiler at github.com/cygx/p6-tinycc/blob/mas...example.p6||19:58|
|precompiling that is where the SORRY got triggered|
|nwc10||jnthn: I've looked at the source code and I can't work out what went wrong from code inspection|
|jnthn||nwc10: I've an idea.||19:59|
|nwc10||I'm guessing that the hash manages to get freed before the iterator|
19:59 Ven joined
|nwc10||I might be about to go to bed, so may not be around if you have a fix to be tested.||20:00|
|jnthn||nwc10: Yeah, it's somethin glike that. Looks like we get to keep a bonus bit of state...|
|lizmat: Hm, seems we may have some spectests sensitive to hash ordering...||20:01|
|lizmat||yeah, that was bound to happen||20:02|
|dalek||arVM: ea6fe32 | jnthn++ | src/6model/reprs/MVMIter. (2 files):
Re-implement MVMIter on hashes using buckets.
|lizmat||ah, just ignore them for now, I will take care of them|
|Sure, well, may not bump MOAR_REVISION/NQP_REVISION tonight anyways.||20:03|
|I need to fix the ASAN barf nwc10++ reported|
|lizmat||those tests shouldn't be a reason not to bump, the ASAN barf is :-)|
|Well, more important is "can I remove the whole linked list thingy"|
|(Which is the goal of all this)||20:04|
|lizmat||jnthn: I'll look at the tests after you bump NQP, is that ok?||20:06|
|or if you could gist the output, that would be great|
|jnthn||lizmat: Sure, though I may get to them anyway to make sure there's nothing more sinister than hash order going on|
20:09 Ven joined
|dalek||arVM: ad17517 | jnthn++ | 3rdparty/uthash.h:
Toss HASH_FSCK addition order check.
This gets rid of the easiest thing depending on ->prev and ->next.
|arVM: 282861f | jnthn++ | 3rdparty/uthash.h:
Use a simpler "deleting the last item" check.
Of note, one that doesn't depend on ->next and ->prev.
20:17 Ven joined 20:34 vendethiel joined
|dalek||arVM: 6774020 | jnthn++ | 3rdparty/uthash.h:
Eliminate double-linked-list in hashes.
With this, we save 16 bytes per hash entry (on 64-bit platforms) and a further 8 bytes per hash. Halve those for 32-bit.
|jnthn||No particular change in instructions executed, but a nice memory saving of course. :)||20:49|
|lizmat||16 bytes / hash entry: how many is it now?||20:50|
|(just prepping info for timotimo's P6W :-)||20:51|
|jnthn||I see 4 pointers and 2 32-bit ints still||20:53|
|Which isn't great, but this is a step in the right direction.|
|dalek||arVM: 0e6e0b6 | jnthn++ | 3rdparty/uthash.h:
Toss an outdated comment.
|jnthn||Getting tired now, so I think I'll fix the ASAN thingy in the morning.||21:00|
21:09 cygx left 22:17 Ven joined
|timotimo||i see about 1.5 megabytes less maxresidentk for a newer rakudo vs an older rakudo||22:35|
|to be exact: f4ff0a8..56cae10 nom 547d08a..18ece1f master (nqp) f04b1d8..0e6e0b6 master (moarvm)||22:36|
22:36 Ven joined
|timotimo||that contains both the "re-use string heap from comp-unit" and the "remove doubly-linked list from hashes" commits||22:39|
23:06 vendethiel joined