00:06 japhb joined 00:26 vendethiel joined 01:48 ilbot3 joined 02:28 vendethiel joined 03:28 vendethiel joined 03:56 vendethiel joined 05:33 vendethiel joined 07:35 zakharyas joined 08:22 rurban joined 09:53 brrt joined
brrt \o 09:57
jnthn o/ brrt 10:29
dalek arVM/even-moar-jit: be79d75 | brrt++ | / (2 files):
Add generic macro facility to tree expr builder

Use any (constant) macro expression inline in the template, not just offsetof and sizeof. The use of a macro is signaled by the
  '&' prefix (like (&offsetof MVMException body.category)).		 10:32
brrt blog post is almost done 10:33
ehm, i was wondering
should i also keep a report on how much hours i've worked in a given time? or is that specific to your arrangment with tpf?
jnthn No, your grant is goal-based 10:41
Mine is time-based, which is why I keep a time report 10:42
brrt very well :-) 10:59
phew, published 11:13
jnthn yay, lunchtime reading for me :)
brrt brrt-to-the-future.blogspot.nl/2015...diate.html i hope it makes sense at all
i'll also be having lunch
timotimo aah good old s-exprs 11:17
good for everything, basically
brrt right :-) 11:21
timotimo ", they are also compact they represent trees without any ambiguity" this grammar has breaken 11:22
"Next week I plan finish" also 11:23
brrt yes
many broke
fortunately
timotimo i'm looking forward to more progress :)
brrt there is an edit button
timotimo i don't see it :P
also, blogspot is quite insistent on giving me the dutch interface
brrt fixedit 11:24
hmm that's weird
i get the american interface
timotimo haha
that's fantastic
hm, is const_i16 actually a thing? or did you mean const_i64_16 (in exprlist) 11:25
brrt well, i have to deal with it
hmmmm
that is a good question
that should've died if it isn't
but it didn't die
actually, const_i16 is the thing 11:26
timotimo OK
i think const_i16 is just not implemented in moar
brrt aha
jnthn brrt++ # post 11:27
And JIT work :)
brrt :-) thanks
jnthn brrt: I'd quite curious to see what the API for REPRs/ext-ops to plug in to this will be :)
brrt well, i think the simplest thing would be something in the terms of the spesh repr extension 11:29
ehm, i mean something like that
timotimo would we be exposing the exprlist parser to "users" of moarvm? 11:30
brrt so a function void (*jit)(MVMThreadContext *tc, MVMSpeshIns *ins, MVMJitCompiler *jc) - or more parameters as needed
and the exprlist parser could then be used to generate a repr-specific template mapping 11:31
timotimo and the exprlist parser would generate a .c or .h file that just gets its data linked with the extop library?
brrt aye
remind me to add that to the Makefile 11:32
dalek arVM/even-moar-jit: 7ada44a | brrt++ | / (3 files):
Add exprlist compilation to makefile
 11:36
brrt now i'm really going to have lunch
bbl :-)
jnthn Have a good lunch! 11:37
jnthn ate lunch and feels less crappy now 11:38
(woke up with a clogged nose, unhappy eyes and ears, etc...)
timotimo do you also get that symptom where it feels like the roof of your mouth is itchy? 11:40
jnthn Not so much that 11:41
I mostly feel like my whole head is clogged up
timotimo ugh
jnthn Kinda like a head ache, but not really ache. 11:42
Though today I don't have that bit so much
Righty, back to the multi-dim stuff 11:49
Hopefully my brain isn't too dim to work on it today... )) 11:50
timotimo hm, it's kinda hard to weigh the cost/benefit to working on something a bit easier when your head is clogged like that 11:51
jnthn: how would you like an addition to the profiler that measures how many objects something allocates die young vs go into the old generation? 11:52
jnthn This morning I mostly hid in bed and prepared for tomorrow's travel... This afternoon my head's a decent bit clearer 11:53
But I did the design work for this stuff earlier in the week, so by now I mostly need to "just" write correct tests and C code :)
timotimo: I'm not sure if we have the data to do that?
Uh, or can easily have... 11:54
timotimo i'd have to build it
i imagine i'd have a list of pointers to things allocated in the nursery recently that the "log allocation" op would fill up
jnthn Hmm 11:55
timotimo then when the nursery gets collected another step would go through all of these objects and check if they were blessed with a forwarder or not
i've got a piece of code right now that surprised me by pretty much linearly increasing its memory usage
having an overview of what objects make it to the old generation might help zone in on what gets allocated but not destroyed 11:56
jnthn One other thing I wish we had is a heap blame
timotimo fwiw, i wouldn't track objects into the old gen to find out if they die before the process ends; i think the administrative overhead may be way too much 11:57
jnthn So we can set an option where at each GC, it spits out what objects are still alive
And where they were allocated
Well, I think we can build things like it, but I don't think it should be on by default
timotimo correct
jnthn In fact, it could be an entirely different kind of profile
timotimo does that refer only to heap blame or also to my life-expectancy-tracking? 11:58
jnthn Depends how expensive the latter is
One thing people like about the profiler today is that the overhead of using it is quite small
And I think that's a good default
timotimo agreed 11:59
jnthn The other problem is that the more things you track, the more it distorts the measurements
timotimo the other day the thought crossed my mind that perhaps the jit could emit better code for logging instructions, but they are quite likely way too complex to improve very much 12:00
jnthn aye
timotimo i think we want to have the "kind of profile" configurable anyway; people have been wishing for a coverage report and probably also a line-based rather than block-based profile
jnthn For example, allocation logging means that allocations are more costly, so the timings get a little distorted for things that allocate
Yeah 12:01
My eventual goal is to have a more general instrumentation framework
timotimo i was quite annoyed when i built the gdb heap analyzer that accessing stuff through gdb via the python api was so darned slow
i was considering building a slim C function that iterates through the nursery and collects things like sizes of objects and REPRs
jnthn That lets you set up "hooks" at NQP level 12:02
Or even Perl 6 level
timotimo oh yeah, that'd be amazing
jnthn It'd be near useless for profiling due to overhead, but for coverage it'd be neat.
timotimo definitely
and for crazy optimization experiments
jnthn Yeah, I was pondering building it as a surprise for YAPC...oh wait... :P
timotimo that means you've already got an almost working prototype? ;)
jnthn heh, no :P
timotimo isn't yapc soon? no, wait, wasn't it recently?
jnthn YAPC::EU 12:03
The Granada one
timotimo mhm
jnthn Damn am I finding an air-con'd hotel for that...
timotimo thoughts of "how do we run the user supplied code without accidentally recursing infinitely" run across my mind
jnthn Yeah, that's an issue ;)
timotimo it could be a per-thread thing
in order to not have nested runloops, it'd probably best have its own thread anyway 12:04
hm, but one good thing about spesh and profiling is that we allocate the stuff needed for spesh in the same thread that has its code improved upon
... but when we run into the user-supplied instrumentation thing, that thread necessarily blocks anyway 12:05
i was about to suggest using nativecall rather than ops to communicate with the API moarvm offers for instrumentation, but there's the ThreadContext thing ... :\ 12:07
on the other hand, building an op that offers the MVMThreadContext to user code could be a whole lot of fun :D
the kind of fun where you break things
all the time
anyway, this may be for another time 12:08
good luck with multidim packed arrays
looking forward to it :)
dalek arVM: 08ed06b | jnthn++ | / (6 files):
Stub in MultiDimArray REPR.
 12:49
jnthn Note how the number of dimensions is part of the type, to aid efficient code generation. 13:01
(The JIT can know how many dimensions there are, and just emit multiplications) 13:02
13:14 brrt joined
timotimo digs into the lifetime tracking for profiling a little bit 13:49
brrt ooh, i like that, dim arrays with known dimensions 13:55
although you immediately find me at a disadvantage 13:58
timotimo hm? 13:59
because you have to make the jit aware of that soon?
brrt no, not soon
but if you have n dimensions you'll typically need n multiplications to find the index 14:00
which means you need a repeating construct
in generating the code
i haven't built that in and am not intending to 14:01
jnthn brrt: No, no
brrt because complex
jnthn We *statically* know how many dimensions there will be
brrt getlex needs it too, btw
jnthn At JIT time
At least, for specialized code
brrt yes :-) but my simple fill-in scheme isn't prepared to deal with that, is what i'm saying 14:02
jnthn Oh
brrt it's no big deal
jnthn We can't just produce a list of multiplications?
brrt for these cases we can do it by hand
yes, of course, i'm just saying it'll need special-casing :-)
anyone know of a good accessiblity-aware editor for linux? 14:03
as in, in emacs zoom doesn't follow my cursor
it does in other apps
timotimo i wonder if the several Qt-based code editors (Kate, Qt Creator, ...) are better at this 14:04
brrt hmm 14:06
gedit works
haven't used that for ages
dalek arVM: e6bcc65 | jnthn++ | src/ (2 files):
Fix name-o.
 14:07
arVM: 4528932 | jnthn++ | src/ (3 files):
Implement MultiDimArray REPR compose and allocate.
timotimo gedit is all right 14:08
MoarVMVM :)
brrt yes, gedit is quite acceptable, now that i've turned tabs to 4 spaces :-) 14:12
timotimo last time i looked, geany seemed quite good 14:13
jnthn: do you think the very end of MVM_gc_collect would be a good place to analyze the nursery's contents for dead/promoted objects? 14:27
jnthn timotimo: Sorta kinda but... 14:30
Look carefully in orchestrate.c
We actually make multiple calls to that
It's a parallel GC and it's not until a certain point you have "consensus" about what is actually going away
timotimo mhh, right
jnthn It does work passing between threads 14:31
timotimo i saw that; i was hoping it'd be enough to put the code below the "pass any leftover work to other threads"
point
hm. the contents of the old nursery get kept around somewhat long, no? 14:35
jnthn After consensus
timotimo perhaps i ought to put the analysis code right at the end of the enter_from_* subs?
jnthn Actually just before the MVM_gc_collect_free_nursery_uncopied calls is probably your best bet
Then in the parallel case you'll potentially be doing the analysis in parallel too... :) 14:36
timotimo ah, that's after "finish_gc" 14:37
that's the position you mean, right? outside the for loop that goes through the gc_work_count'ed list
jnthn Correct
If you imagine Moar's collector was a mark and sweep (which it's not, but it holds up here) then we stop the world to mark (in parallel), but the threads are then freed to go their own way in sweeping 14:38
timotimo hum, actually, i may have to do it in that loop so i get all threads' ptds, but the profiler dies in multithreaded code anyway 14:39
jnthn Oh?
I never tested it
timotimo ah, so run_gc is run on every thread?
jnthn I designed it so we could get there
Yeah
timotimo OK, that means i'll do it outside the for loop
jnthn No on, you want it in the for loop 14:40
Right before the call to MVM_gc_collect_free_nursery_uncopied
timotimo right, i was about to complain that i don't get it ;)
jnthn I'm just saying that at that point, we're not in "stop the world" mode any more
And the data you're looking at is stable
timotimo OK
jnthn So it's a good time.
timotimo checking for prof_data to be NULL is the right indication for "are we profiling"? or would i look in instance somewhere? 14:41
yeah, instance has a flag for it
that'll be better
jnthn I think there's a function for it even
timotimo well, ack spits out a whole lot of if(tc->instance->profiling) { ... } 14:43
so if there's a convenience function for it, we don't use it consistently yet
dalek arVM: 5652ca5 | jnthn++ | src/6model/reprs/MultiDimArray.c:
Fill out GC bits of MultiDimArray REPR.
jnthn *nod* 14:44
There probably ain't one yet though
*then
If we add one it'd want to be an MVM_STATIC_INLINE
timotimo yup
oh, now it gets funny 14:46
since i ought to track objects through more than one nursery, the list of objects to track can get fragmented
i may have to build a free list for it, which wouldn't just work with pointers, as it gets realloc'd frequently
so a linear scan may be necessary, but ... ugh 14:47
or a compaction after the scan
i don't yet have a clue how many objects would end up getting tracked
dalek arVM: 1077907 | jnthn++ | src/6model/reprs/MultiDimArray.c:
Implement introspective bits of MultiDimArray.
 14:58
brrt jnthn: what is operandd type mask 14:59
jnthn brrt: Context?
timotimo masks the flags for whether it's a lexical, literal, register, ...
jnthn Other than "a typo for operand"
timotimo because the flags also have write/read flags
brrt aha
nm
timotimo you can check the big piece of spesh dump that dumps operands 15:00
brrt it's the mask for the operand type field
timotimo spesh dump code
thingie
jnthn Oh, in interp.h...
brrt yes, there
:-)
brrt was wondering what a mask type would be
timotimo and what kind of person would wear that kind of mask 15:01
jnthn Yeah, the lower 3 bits used for literal/lexical/register read/write
And the next 5 bits for what type of register it is
Short break; bbs 15:04
timotimo oh, damn. i'm refering to the MVMProfileAllocationCount by pointer, but that's an address into a region tthat gets realloc'd 15:05
brrt well 15:06
why do you to that then :-P
timotimo now i have to schlepp around one more field ... this may be the moment where it'd be worth building a struct rather than having the arrays "flat" in the struct :\
15:09 vendethiel joined
timotimo i'm just doing a PoC at the moment anyway 15:12
at least it feels like that
jnthn Darn this hot weather. plz can haz winter now? 15:25
brrt insufficent winter available 15:26
timotimo people are locking away important cold in their fridges
if only everybody could open up their fridge, the world could be saved from global warming
lizmat Must. Keep. Breathing.
brrt nice timo :-P 15:27
lizmat our neighbour has a new airco, with the exhaust hanging from an half opened window pointing downward
brrt who has an airco in nl
seriously
who does that
that makes no sense 15:28
lizmat more and more people, with the odds of a heatwave going up to once every 3 years
brrt for the one week in which it is warm in the year
well, that's... true-ish, i guess
400 ppm and counting, folks
jnthn My wife was asking if our place here in Prague has aircon. I showed her the climate chart for here, at which point she agreed it's probably useful for a handful of times a year. :) 15:29
(The hottest months are July and August, and the average daily high is only 23C) 15:30
nwc10 jnthn: how thick are the walls? (how old is the building?)
TimToady and which floor are you on? 15:31
jnthn 3rd floor (or 2nd the British way)
nwc10: Given I don't hear a peep from neighbors, or much from outside if I close the windows, I figure fairly thick. :)
It *is* south-facing, however.
timotimo oh lord ... could it be my ProfileCallNodes are moving from below me because profile_log_enter reallocs its succ array >_< 15:32
nwc10 if it's altbau (and dates back to the Austro-Hungrian emptire times) then I suspect that the walls are thick enough to ignore summer 15:33
I can haz internetz (2.0) sort off
sort of
15:34 vendethiel joined
jnthn Today is the first day that it's actually Just Too Hot in here, and it's more than 10C above the average daily high today. 15:34
nwc10 in that, rather than my laptop on the USB dongle, I now have the Pi doing it, and NAT
jnthn (WU claims currently 34.7C)
lizmat "only" 33.7 here 15:35
jnthn Still very urgh
timotimo is the only thing i can do to not lose my sanity to give a PCN a list of things that point to it or alternatively turn its succ array into an array of pointers instead? >_<
jnthn hopes the claims of aircon where he's spending the next week actually hold up :) 15:36
timotimo we have above 35 here according to the weather station website thingie
wetterstationen.meteomedia.de/?stat...vorhersage
and that's going to stay for tomorrow and the day after that apparentlty
jnthn Righty, let's try and crank out a bit more of this multi-dim impl 15:37
Darn. Now I have to do multiplication rather than just pointer shuffling...
timotimo alternatively decouple the measurements from the individual call nodes to get just a global overview of how many objects of what type were "killed" in any given gc run
jnthn: it could prove valuable to be able to supply different "strides" for arrays somehow 15:38
is that part of the current design?
jnthn timotimo: Considered keeping a "shadow nursery" per thread?
timotimo: No, I'm just going with C layout semantics, which is what S09 talks about
timotimo hmm
brrt what is internet 2.0
timotimo i'm not sure if a shadow nursery would help me here
jnthn (by shadow nursery I mean that you keep a copy of the allocations "in order", and then you can go thorugh the real nursery and compact the shadow one as you analyze it) 15:39
timotimo brrt: it's basically identical to "web 2.0", but it also shows that you have no idea that the internet has more services than just HTTP on port 80
jnthn Then you're linear-scanning the two
timotimo the problem i'm struggling with here is how to attribute a given dead object to the process call node it was allocated in
scanning the objects i have is no big problem; my current implementation is somewhat like a shadow nursery already 15:40
jnthn Oh, and call nodes can move, or?
timotimo but it only contains things that were allocated together with a given log allocation instruction
yeah, log_enter can realloc the succ array of child nodes and i take a pointer to a PCN :(
more exactly: the succ array is just all pcns in order, rather than a list of pointers to the pcn objects 15:41
jnthn Really?
/* Successor nodes so far. */
MVMProfileCallNode **succ;
It doesn't look that way, that's a **
Also if we did that surely this bit doesn't end well:
/* The node in the profiling call graph that we came from. */
MVMProfileCallNode *pred;
timotimo huh 15:42
weird, i must hav emisread
jnthn And the realloc I see is certainly about the pointer array
pred->succ = MVM_realloc(pred->succ,
pred->alloc_succ * sizeof(MVMProfileCallNode *));
That sizeof is size of a pointer
timotimo yeah, i did misread
jnthn So you're safe :)
timotimo so let's see what i'm still doing wrong
ah, oops 15:43
copy-pasted the realloc line and forgot to update the first parameter 15:44
now it runs %) 15:45
um ... huh? here's an instance where "dead_before_gen2" ends up a very, very big number, but the actual number allocated is 5 15:51
m: say 8590065665.base(16) 15:52
camelia rakudo-moar add282: OUTPUTĀ«200020001ā¤Ā»
timotimo m: say 8590065665.base(2) 15:53
camelia rakudo-moar add282: OUTPUTĀ«1000000000000000100000000000000001ā¤Ā»
timotimo multiple occurences of that stuff not going so well 15:54
ah, i forgot to check if things got moved to the old gen
that may cause things to get counted a whole bunch of times 15:55
damn, fixing that didn't get rid of the bogus values yet :( 16:02
jnthn I guess it's not simply forgetting to obtain zeroed memory? 16:04
timotimo oh, huh 16:05
it *could* be i have to manually null that
i thought i saw calloc be used for that
oh, i put the code in a bogus location, too :) 16:06
dalek arVM: a24501f | jnthn++ | src/6model/reprs/MultiDimArray.c:
Implement MultiDimArray setting dimensions.

This is also where the memory allocation takes place. Take some care to be memory-safe in the face of multi-threaded abuse.
timotimo the code above my code looked so weird, now iknow why %) 16:07
timotimo puts a goto into the code and hopes no velociraptor comes to nom him 16:08
don't even need it
cool, seems to work now 16:10
on to the HTML portion
dalek arVM/even-moar-jit: b5155eb | brrt++ | / (5 files):
Nearly complete template-based building

Added very minimal IR definitions; likely needed to expand quite a bit in the future. Also tied exprlist compilation into expr.c building.		 16:30
brrt up next: roots
jnthn brrt++ # progress :) 16:31
brrt :-)
i'm perosnally very happy with the fact that the template application routine is as simple as it is 16:32
and... roots needs a merge operation 16:33
oh, and i guess i've forgotten about adding stores, too 16:35
hmm
if i'm doing that, roots merging is a lot simpler, methinks 16:36
no
i'm wrong
but i need to have dinner anyway :-)
jnthn :)
brrt bbl 16:38
timotimo higher order macros?! 16:49
16:53 vendethiel joined
dalek arVM: 9422c59 | jnthn++ | src/6model/reprs/MultiDimArray.c:
Implement read/write to multi-dim array.
 17:02
jnthn Didn't write the test coverage yet, but this should also work for packed multi-dimension integer arrays 17:05
timotimo something i'd love to use in my current script :)
jnthn Yeah, once I get the VM level nailed there'll still be the Perl 6 level to do :)
timotimo exposing that to perl6 code will take a bit yet, though
yeah
jnthn anyway, shopping for nom, eating nom, etc. 17:06
timotimo yup :)
bonne nom
18:30 zakharyas joined
jnthn nam :) 18:33
jnthn figures he'll do his packing now, in hope that lower temperatures into the evening will make hacking on stuff a bit more easier :) 18:35
dalek arVM/early_death_percentage: 6ea2e55 | timotimo++ | src/ (4 files):
track freshly allocated objects to see how early they die
 19:07
jnthn Righty, let's write some more code... 19:31
dalek arVM: c3de73a | jnthn++ | src/6model/reprs/MultiDimArray.c:
Fix a silly thinko in index calculation.
 19:49
arVM/even-moar-jit: 8a6a6c7 | brrt++ | src/jit/expr.c:
Add roots and end-of-block stores

Roots are the terminals of the tree. The code generator should evaluate each root in order. At the end of a block, values are spilled to memory (that is, until we implement register allocation). These spills also become roots. This should ensure that the whole basic block is eventually compiled.		 20:03
arVM: 69aeeee | jnthn++ | src/6model/reprs/MultiDimArray.c:
Refuse dimension-changing operations.
 20:04
jnthn aeeee!!! 20:05
tadzik aeeee!!! www.youtube.com/watch?v=eh7lp9umG2I 20:11
dalek arVM: 14c3a50 | jnthn++ | src/6model/reprs/MultiDimArray.c:
Make at_pos and bind_pos work on 1D MultiDimArray.

This will ease usage of it when we simply want to have fixed-size 1D arrays.		 20:30
timotimo that's a nice commit short-id 20:33
jnthn tadzik: I didn't last 10 hours of it :P 20:43
timotimo This video contains content from Classic Media Ltd, who has blocked it in your country on copyright grounds. 20:44
^- god damn it
but other versions are still available
like the live-action version
dalek arVM: fa318d6 | jnthn++ | src/6model/reprs/MultiDimArray.c:
Implement elems/setelems on MultiDimArray.

Again, to ease use as a 1D fixed size array.		 20:47
jnthn Well, that's got that REPR filled out... 20:48
Oh, except the serialize/deserialize bits 20:49
Damn, the hot has been draining. Think I leave it there for tonight 20:56
timotimo the hot is draining me a lot, too 20:57
jnthn Already time for the next grant weekly too 20:58
Mebbe I'll work on that on the place tomorrow :)
*plane
21:09 TimToady joined 21:16 TEttinger joined 22:48 vendethiel joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!