03:57 vendethiel joined 07:56 rurban joined 08:19 Ven joined
dalek arVM: 15908a2 | jnthn++ | src/gc/roots.c:
Mark the cached backend config hash.
 08:42
nwc10 jnthn: much rakduo SEGVing on linux. at this line of moar.c: 08:46
MVM_interp_run(tc, &toplevel_initial_invoke, start_frame);
seems to be that the address of start_frame has the lowest bit set
jnthn nwc10: Did you have 15908a2? 08:47
08:47 Ven joined
nwc10 I think I was on 15908a2^ 08:48
jnthn OK, but...weird. I've a build on Linux here too and all's well with latest of everything
nwc10 I'm trying to figure out what caused it 08:49
jnthn m: say 9611866 - 9552454 09:36
camelia rakudo-moar f3ecf8: OUTPUT«59412␤»
jnthn Another 60KB off CORE.setting 09:41
That's 60KB less *bytecode* to verify and interpret 09:42
nwc10 with the cached backend config? 09:43
jnthn No 09:44
That was a smaller win (about 1.5 million less CPU instructions)
Well, a different kind of win, rather.
nwc10 spectest seems to be happy with Moar at 15908a2 09:45
(Rather than 15908a2^)
jnthn Oh, I get a nice CPU win too 09:47
m: say 541034601 / 552393040 09:48
camelia rakudo-moar f3ecf8: OUTPUT«0.9794377587␤»
jnthn Another 2% off startup. I'll take that.
dalek arVM: 0acd74b | jnthn++ | src/ (6 files):
Dual-purpose comp unit string heap as SC one.

This means we don't have to construct an array of strings in the order the deserialization blob wants them, and can instead just point the deserialization at the string heap of the compilation unit. When the NQP change to take advantage of this is applied, we knock 60KB off the size of CORE.setting, and save 2% off startup time.		 09:51
jnthn m: say 541034601 / 644090520 09:54
camelia rakudo-moar f3ecf8: OUTPUT«0.8399977708␤»
jnthn Now running only 84% of the instructions at startup we used to compare to yesterday.
nwc10 cool. 10:00
dalek arVM: 5179453 | jnthn++ | src/core/bytecode.c:
Re-order bytecode read struct to be smaller.
 10:06
arVM: 48171d9 | jnthn++ | src/core/bytecode.c:
Don't repeatedly calculate limit in bytecode read.

Also mark it MVM_INLINE_STATIC and remove a bogus NULL check.
jnthn m: say 540112560 / 541034601 # a tiny but easy win 10:10
camelia rakudo-moar f3ecf8: OUTPUT«0.9982957818␤»
10:18 Ven joined 11:43 Ven joined 12:14 Ven joined 12:20 rurban joined
timotimo cool, i like these changes 12:57
13:17 vendethiel joined 14:02 vendethiel joined 14:39 mj41 joined 14:56 zakharyas joined
dalek arVM: 6b7a61d | jnthn++ | 3rdparty/uthash.h:
Toss some bits of ut_hash we don't use.

As part of preparations for more extensively modifying it.		 15:18
arVM: cf4ef03 | jnthn++ | 3rdparty/uthash.h:
Toss hash bloom test bits we don't use.
 15:26
timotimo ooooh 15:27
if we heavily modify it, maybe we'll end up having MVMString objects that actually store latin1 if they can
er, MVMString isn't an "object"
but you know what i me
mean*
jnthn It is an object as far as the GC cares :P 15:28
But yeah, that's kinda where I'm going :)
timotimo yus! 15:30
memory usage could go down a whole lot from this
on the weekly, someone commented having NFG strings for everything could lead to problems because it balloons up the memory usage of a single character even if it could be stored in latin1 15:31
like if you analyze log files or something and the log file happens to be a gig in size
jnthn If you're reading it all into memory you may already have issues ;) 15:32
But yeah :)
timotimo fair enough
jnthn I think if you explicitly decode the thing as latin-1 then we can probably get it stored byte size. 15:34
timotimo mhm 15:35
since we have ropes, we could also decode until we find something that latin1 can't handle and switch over and have a two-part string
except if the latin-1-codable prefix is too short
dalek arVM: 315aac7 | jnthn++ | 3rdparty/uthash.h:
Toss unused hash signature.

We don't make any use of this, so it's just 32 wasted bits.		 15:36
jnthn timotimo: Yeah, I'd pondered that too :)
timotimo maybe i ought to freshen up the heap analyzer plugin to gdb and have a look at what's on the heap these days 15:37
last time i looked we had multiple copies of the same MVMString in the nursery for a big amount of strings 15:38
i wonder where that came from and if it's still there
15:45 Ven joined
dalek arVM: 77e3fbc | jnthn++ | Configure.pl:
Optimize at level 3 by default.

We've been at 2 for a good while without issues; let's bump it a tad further. Doesn't make a big difference, but every little helps.		 15:46
arVM: 42d7dd8 | jnthn++ | 3rdparty/uthash.h:
Toss even more hash bits we don't use.
 15:58
16:03 Ven joined
timotimo i've been on -O3 for a long time now 16:14
(and i even -march=native)
jnthn This is the best error I've got out of a C compiler in ages! 16:26
C2064: term does not evaluate to a function taking -1000 arguments
arnsholt \o/ 16:27
timotimo m) 16:30
jnthn is further hacking up the code once known as UT hash 16:32
I'm currently working on getting rid of the linked list for the order stuff was added in.
Which'll save us 16 bytes (on 32-bit machines) per item put into a hash.
I *think* it should be do-able without a total re-write of the thing. I hope. :) 16:33
16:57 rurban joined
nwc10 sort of a too-late question now - you started on the version we had, not the most recent (and now abandoned) version upstream? 17:25
jnthn nwc10: No 'cus we already have some more minor modifications, though looking at the change log I think pretty much all that we'd have missed was in the code I tossed anyway 17:27
Seems I've managed to re-work the HASH_ITER macro to go through the buckets rather than needing the ordered linked list. 17:29
jnthn spectests that piece
I think there's another iteration path I'll need to change. 17:30
And then can see if tossing the addition order linked list is as easy as I hpe.
*hoe
*hope
...yeah, apparently it's dinner time too :)
18:10 vendethiel joined 18:44 vendethiel joined
dalek arVM: 7c1aaf7 | jnthn++ | / (9 files):
Re-implement HASH_ITER in terms of buckets.

This is the first step in removing the use of the double-linked list that maintains insertion order, which from a NQP/Perl 6 perspective is simply time/memory overhead that we don't need.		 18:49
lizmat which means we finally get randomly ordered hashes ? 18:52
jnthn lizmat: I don't think MVMIter uses the macro, so that's next in line 19:03
But yeah, we'll lose insertion order
lizmat yes! :-)
jnthn "randomly" is maybe a bit strong :)
lizmat wonder how much this will break in the ecosystem :-)
jnthn I hope not much given that JVM hasn't given the ordering 19:04
lizmat weill, it'll allow some .pick(*) in some .perl's, to be removed 19:06
timotimo keys will be placed pseudorandomly depending on their values as well as insertion order 19:13
OSLT
we'll have to do a bit of extra work to make the order different on every execution of the same script 19:16
19:17 Ven joined
lizmat I don't think that's necessary just yet 19:17
jnthn timotimo: We'll likely have to do that to avoid security issues
lizmat well, before 6.0 sure :) 19:18
jnthn Thankfully there's some good knowledge on these issues in the Perl community
lizmat true, but it's also a source of discussion, and sometimes feuds :-( let's not go there :-) 19:19
timotimo yes
jnthn It's fine, I only have to listen to the people who are right. :P 19:25
19:25 zakharyas joined
lizmat just like picking the right standard :-) 19:25
hmmm... I just realized this may break typed hashes 19:26
as it uses 2 hashes: one for the keys and one for the values
jnthn Should be fine, I think... 19:30
19:31 Ven joined
lizmat yeah, HashIter is doing a lookup 19:35
jnthn: do you have any idea why it was done that way, rather than having a single hash with Pairs (original object => value) ?? 19:36
19:36 Ven joined
lizmat generation of .pair/.kv and friends would be comparable, without the need of keeping a whole 2nd hash with identical keys 19:37
fwiw, that's how I implemented bags/mixes
jnthn lizmat: Because so far we've only got (at the lower level) support for hashes with string keys 19:39
lizmat yes, I get that
but why not the approach of StringKey => ( Key => Value ) 19:40
instead of (StringKey => Key), (StringKey => Value)
internally, I mean
19:41 Ven joined
lizmat the natural form of a Hash is a list of pairs, which is easily generated from the former setup :-) 19:41
jnthn Ah, I see your point
Not sure; what you're suggesting could be better
lizmat and is proven, as bags/mixes are implemented that way :-) 19:42
jnthn *nod*
nwc10 jnthn: ASAN! paste.scsys.co.uk/475870 19:43
19:46 vendethiel joined
jnthn urgh 19:47
19:56 cygx joined
cygx jnthn: while you're working in the general vincinity - I just got a ===SORRY!=== Missing serialize REPR function for REPR VMIter 19:56
jnthn cygx: That's probably quite orthogonal. 19:57
cygx jnthn: just saw an MVMIter in the logs
in related news: hello world from the Tiny C Compiler at github.com/cygx/p6-tinycc/blob/mas...example.p6 19:58
precompiling that is where the SORRY got triggered
nwc10 jnthn: I've looked at the source code and I can't work out what went wrong from code inspection
jnthn nwc10: I've an idea. 19:59
nwc10 I'm guessing that the hash manages to get freed before the iterator
19:59 Ven joined
nwc10 I might be about to go to bed, so may not be around if you have a fix to be tested. 20:00
jnthn nwc10: Yeah, it's somethin glike that. Looks like we get to keep a bonus bit of state...
lizmat: Hm, seems we may have some spectests sensitive to hash ordering... 20:01
lizmat yeah, that was bound to happen 20:02
many ?
dalek arVM: ea6fe32 | jnthn++ | src/6model/reprs/MVMIter. (2 files):
Re-implement MVMIter on hashes using buckets.
jnthn 3
S32-hash/delete-adverb.t
S32-list/roll.t
lizmat ah, just ignore them for now, I will take care of them
jnthn S05-capture/caps.rakudo.moar
Sure, well, may not bump MOAR_REVISION/NQP_REVISION tonight anyways. 20:03
I need to fix the ASAN barf nwc10++ reported
lizmat those tests shouldn't be a reason not to bump, the ASAN barf is :-)
jnthn *nod*
Well, more important is "can I remove the whole linked list thingy"
(Which is the goal of all this) 20:04
lizmat jnthn: I'll look at the tests after you bump NQP, is that ok? 20:06
or if you could gist the output, that would be great
jnthn lizmat: Sure, though I may get to them anyway to make sure there's nothing more sinister than hash order going on
20:09 Ven joined
dalek arVM: ad17517 | jnthn++ | 3rdparty/uthash.h:
Toss HASH_FSCK addition order check.

This gets rid of the easiest thing depending on ->prev and ->next.		 20:14
arVM: 282861f | jnthn++ | 3rdparty/uthash.h:
Use a simpler "deleting the last item" check.

Of note, one that doesn't depend on ->next and ->prev.
20:17 Ven joined 20:34 vendethiel joined
dalek arVM: 6774020 | jnthn++ | 3rdparty/uthash.h:
Eliminate double-linked-list in hashes.

With this, we save 16 bytes per hash entry (on 64-bit platforms) and a further 8 bytes per hash. Halve those for 32-bit.		 20:43
jnthn No particular change in instructions executed, but a nice memory saving of course. :) 20:49
lizmat 16 bytes / hash entry: how many is it now? 20:50
(just prepping info for timotimo's P6W :-) 20:51
jnthn I see 4 pointers and 2 32-bit ints still 20:53
Which isn't great, but this is a step in the right direction.
dalek arVM: 0e6e0b6 | jnthn++ | 3rdparty/uthash.h:
Toss an outdated comment.
 20:57
jnthn Getting tired now, so I think I'll fix the ASAN thingy in the morning. 21:00
21:09 cygx left 22:17 Ven joined
timotimo i see about 1.5 megabytes less maxresidentk for a newer rakudo vs an older rakudo 22:35
to be exact: f4ff0a8..56cae10 nom 547d08a..18ece1f master (nqp) f04b1d8..0e6e0b6 master (moarvm) 22:36
22:36 Ven joined
timotimo that contains both the "re-use string heap from comp-unit" and the "remove doubly-linked list from hashes" commits 22:39
23:06 vendethiel joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!