timotimo gnite jnthn :) 00:00
01:07 synopsebot6 joined
dalek arVM/moar-gdb.py-python3: 4263b71 | timotimo++ | tools/moar-gdb.py:
[moar-gdb.py] fix up lots of more stuff for py3 and such
 02:26
arVM/moar-gdb.py-python3: 6df338e | timotimo++ | tools/moar-gdb.py:
[moar-gdb.py] throw out "reached" crutch
02:51 colomon joined 07:05 domidumont joined 07:08 domidumont joined 08:00 vendethiel- joined 08:19 TimToady joined 08:32 zakharyas joined 09:09 kjs_ joined
arnsholt jnthn: Python 3 has betterer Unicode, but it's still a bit weird I think 09:24
09:27 vendethiel joined 09:41 kjs_ joined 10:13 vendethiel joined 10:35 JimmyZ_ joined 10:50 kjs_ joined 11:53 vendethiel- joined 12:13 vendethiel joined 12:16 kjs_ joined 12:17 kjs_ joined 12:34 vendethiel- joined 12:52 colomon joined 13:37 kjs_ joined
timotimo jnthn: i see the p6opaque reprdata mallocs/frees a bunch of things separately. like total_attrs times sizeof MVMuint16, then of MVMSTable*, then MVMObject*, then MVMuint16 again 13:58
and then total_attrs + 1 times three MVMuint16s
not only could i make all this use the FSA, i could probably also pack all this stuff into one memory region that gets allocated and freed in one go, and then just store pointers into that region (including some alignment, of course) 13:59
how does that sound?
i suspect that'll also make things a tiny bit more cache-friendly, because then there won't be malloc headers in between those fields 14:01
though i haven't checked if that's actually the case. i could, though!
arnsholt Sounds plausible to me, at least =) 14:04
timotimo also, every REPROps struct is called "this_repr", and gdb only displays that 14:05
how do we feel about going through all the reprs and renaming that variable to hold the name of the actual repr?
that'd make it immediately visible in the debugger 14:06
rather than having to print out the reprops and look for the "name" field
(which, incidentally and thankfully is at the very bottom)
hm. how do i best make this "visible" to the naked eye, what's going on? 14:15
are we against alloca in mvm? 14:24
the only mention i see of it is in spesh dump, where i placed it myself
annoying that "print alignof(...)" doesn't seem to work in the gdb shell 14:34
timotimo went with a little "bump the pointer" allocation scheme with a separate virtual preparing stage and a "realization" stage 14:48
soon i'll find out if it actually works or not
hm. so what FSA should i be using for this, i wonder ... 14:49
i probably have to build a new one?
nope, instance has a "global" one 14:52
hm. the FSA free function wants to be told the size 14:58
jnthn Yes, that you know the size you're freeing is *why* it doesn't have to keep track of it 15:21
That's half the point :)
timotimo oke :)
gets rid of the necessity of headers 15:22
though an expensive search operation could be built to figure out what exact bin something has landed in, and from there "guess" the size
jnthn Right :)
urgh, no
Just stick with malloc if it's too annoying to easily know the size
timotimo nah, it's okay. i put a field into MVMP6OpaqueREPRData 15:26
what type would i use for a memory offset to be added to a pointer?
geekosaur ptrdiff_t? 15:27
timotimo thanks
src/6model/reprs/P6opaque.c:691:90: error: ā€˜MVMP6opaqueREPRData {aka struct MVMP6opaqueREPRData}ā€™ has no member named ā€˜name_to_index_mappinā€™ 15:28
geekosaur is that missing a letter? 15:29
timotimo nah, just mappin' 15:30
for some reason, though, C doesn't accept that name
other fields of this struct: flattened_stable_hoppin' as well as gc_mark_boppin'
man, if only it were easier to get the exact amount of memory being "wasted" by mallocing those small fields 15:32
so i could get a more precise hint as to how much my code would save
d'oh, i'll also have to do this work in deserialize_repr_data 15:33
urgh, i may just have to write the size field into the serialization blob in order to do all of this right 15:35
or i can malloc the data for "in transit" storage and later memcpy it to the right addresses in the fixed_size blob and free the "in transit" stuff in between 15:36
no, there's an MVM_ASSIGN_REF in there that'd probably blow things up
timotimo sits in front of tumble dryer and watches 15:37
perfect representation of thoughts in my head right now
(also it's comfy and warm there)
jnthn Urgh, no, don't store the size in the serialization blob... 15:38
What if we get a bogus file? 15:39
Buffer overrun exploit waiting to happen.
timotimo oh, hmm.
jnthn Not to mention bloating the serializatin blob
timotimo OK, so for the compact blob approach for storing all the data for p6opaquereprdata i'll have to juggle a lot of code around in the deserialize_repr_data function 15:41
jnthn wonders if it'll be worth it... 15:42
timotimo i couldn't come up with something clever to visualize the spread of fields of the reprdata
m: ay minmax 0x24371a0, 0x2437180, 0x2436f70, 0x2436f90, 0x2436fb0, 0x2436fd0, 0x24371c0, 0x24371e0, 0x2437200 15:46
camelia rakudo-moar b9a79e: OUTPUTĀ«===SORRY!=== Error while compiling /tmp/Ay_qY2QGzBā¤Undeclared routine:ā¤ ay used at line 1ā¤ā¤Ā»
timotimo m: say minmax 0x24371a0, 0x2437180, 0x2436f70, 0x2436f90, 0x2436fb0, 0x2436fd0, 0x24371c0, 0x24371e0, 0x2437200
camelia rakudo-moar b9a79e: OUTPUTĀ«37973872..37974528ā¤Ā»
timotimo m: .bounds.fmt("%x").say given minmax 0x24371a0, 0x2437180, 0x2436f70, 0x2436f90, 0x2436fb0, 0x2436fd0, 0x24371c0, 0x24371e0, 0x2437200 15:47
camelia rakudo-moar b9a79e: OUTPUTĀ«2436f70 2437200ā¤Ā»
timotimo m: say [-] @(.bounds) given minmax 0x24371a0, 0x2437180, 0x2436f70, 0x2436f90, 0x2436fb0, 0x2436fd0, 0x24371c0, 0x24371e0, 0x2437200
camelia rakudo-moar b9a79e: OUTPUTĀ«-656ā¤Ā»
timotimo this is the spread of pointers from a reprdata with just one attribute
though to be fair, this includes pointers that my "packed" approach will not put into the package, so let me re-do that list 15:48
m: say [-] @(.bounds) given minmax 0x2437180, 0x2436f70, 0x2436f90, 0x24371a0, 0x2436fd0, 0x24371c0, 0x24371e0, 0x2437200 15:50
camelia rakudo-moar b9a79e: OUTPUTĀ«-656ā¤Ā»
timotimo seems like there's only a single pointer i pasted that wasn't in that bunch
15:51 ggoebel16 joined
timotimo for a P6opaque with 1 num_attributes, we'll build a blob of size 208 15:52
^- i can't tell if this is the same kind of p6opaque as seen above
for a P6opaque with 1 num_attributes, we'll build a blob of size 176
^- but there's also some that'll end up smaller 15:53
for a P6opaque with 10 num_attributes, we'll build a blob of size 464
now, how do i calculate if we'd end up just mallocing that blob because it doesn't fit into a bin? 15:55
we discard the first 3 bits, after that, each value gets its own bin; we hold 96 bins 15:56
so the biggest we'd be having is 63 << 3?
m: say 63 +< 3
camelia rakudo-moar b9a79e: OUTPUTĀ«504ā¤Ā»
timotimo neat. that'll hold many of these blobs
for a P6opaque with 11 num_attributes, we'll build a blob of size 504 15:57
^- up to this
for a P6opaque with 14 num_attributes, we'll build a blob of size 504
^- this one probably doesn't have a big mro or something? 15:58
does that analysis seem sane at all? 15:59
16:02 hoelzro joined, virtualsue joined
timotimo moar-gdb.py should probably learn about the FSA, too, and show some details about that 16:03
jnthn: does MVMP6opaqueREPRData seem like a good candidate to be fixed_size_alloc'd, too? it's 70 bytes big (because i added the size_t fsa_allocated_bytes field at the end) 16:22
i'd think so 16:23
16:26 kjs_ joined
timotimo jnthn: alloca, yay or nay? 16:31
masak .oO( alloc-aye! ) 16:40
jnthn stackoverflow.com/questions/1018853...d-practice has some warnings on it 16:41
timotimo OK, i'll use malloc and free instead 16:42
oh, neat. when deserializing i can even make the storage of the name_to_index_mapping contiguous 16:52
nice. the code in its current state seems valgrind-clean 17:15
alas, it's wrong 17:16
timotimo takes a break 17:17
173,432 bytes in 3,097 blocks are possibly lost in loss record 1,582 of 1,620 / 0x5037B4B: generate_unicode_property_values_hashes (unicode.c:48476) 17:41
is this known? will we clean up our unicode database upon shutdown?
(perhaps this just happens because moar exited with a traceback and failure state?) 17:42
jnthn timotimo: Yeah, I know that one 17:44
One of the things I want to get clean :)
timotimo good to know :)
now i wonder where my code b0rks the compilation 17:45
Cannot look for method 'Num' on a null object - clearly something's been corrupted 17:48
17:56 kjs_ joined
dalek arVM/p6opaque_packed: 3b5a50d | timotimo++ | src/6model/reprs/P6opaque. (2 files):
WIP on having P6opaque store its reprdata in one FSA'd blob

instead of mallocing a bunch of tiny fields for its arrays, which are all likely padded to 32byte sized blocks and include a piece of header info.		 18:19
18:29 domidumont joined
timotimo i resisted the urge to call the branch "p6opaque_paqued" 18:37
18:38 kjs_ joined
ilmari p6opackqued 18:39
19:06 ilbot3 joined
nwc10 timotimo: patch on branch MoarVM/p6opaque_packed ? 19:06
timotimo yup
nwc10 I'll try, but there's rather a lot going on
small minion has just been put into bed, but is not asleep
timotimo oh, okay
nwc10 and is *nearly* capable of climbing out
timotimo don't worry about it, then :)
nwc10 I'll try to get a good look at it 19:07
timotimo i'll put some debugspam into the code to see if things are getting wrongly overwritten
nwc10 but I can't promise a timescale, sorrt
19:17 virtualsue joined 19:31 flussence joined, harrow joined, timotimo joined, BinGOs joined 19:32 geekosaur joined
timotimo nwc10: gist.github.com/timo/bcb99fb065b739edc94e - here's offsets into the memory blob as well as a little hexdump (modified to show nullbytes as underscores ) 19:32
19:33 harrow joined
timotimo it's a little disconcerting to have so many unused nullbytes at the beginning in so many cases 19:34
oh, wait
no, don't wait! 19:35
in any case, i can detect that all values from a section are zeroes and collapse multiple all-zeroes-sections into a single one that all share a pointer 19:36
for ALL ZE GAINS
timotimo makes the hexdump a bit nicer to look at 19:40
updated the gist with a new revision 19:47
and again 19:49
that was fun, but i didn't actually debug anything with that 19:52
now with ` instead of _ for null bytes 19:56
much less noisy
so, who fixes my crappy code now? :D 19:57
dalek arVM/p6opaque_packed: 9a121a4 | timotimo++ | src/6model/reprs/P6opaque.c:
super debug spam: output a hexdump of the packed blob
 20:13
timotimo i'm now building a simple little second branch that just replaces every malloc in there with a fixed_size_alloc, which may still be better than what we had with malloc 20:26
20:27 Ven joined 20:38 zakharyas joined
timotimo dinner dinner 20:55
i wonder why the memory usage of perl6 -e 'say "hi"' is so unstable 21:23
21:27 pyrimidine joined 21:32 ggoebel16 joined 22:14 virtualsue joined 22:57 geekosaur joined 23:13 kjs_ joined 23:41 lizmat joined, khagan joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!