01:13 lizmat joined 01:55 ilbot3 joined
Geth MoarVM: ugexe++ created pull request #679:
Mark GC blocked when acquiring mutex		 03:15
04:19 lizmat_ joined 04:21 nwc10 joined, samcv joined 04:24 nativecallable6 joined, greppable6 joined, releasable6 joined, bisectable6 joined 04:41 ilmari[m] joined 06:47 brrt joined, leont joined
brrt good hi #moarvm 06:48
leont Morning! 06:51
samcv morning!
good hi brrt :-)
okie dokie 06:52
brrt morning samcv, leont
samcv brrt, i was trying to set up the KMP string index so that it would search a partial needle length, so we have max pattern length 06:53
and if it found that pattern, it would then check if the substring of what's left of the needle matched the rest of the haystack. but i was running into issues. maybe i had the numbers off. but it was very frustrating and i gave up after a few hours 06:54
though 8192 is bigger than most needles probably are. and the jump table uses 16bit integers instead of using 32 bit before
brrt uhuh
samcv i doubled the jump table limit and doubled the max pattern length 06:55
few days ao
i'll probably revisit it eventually
brrt no rush :-)
samcv 8192 needle graphemes is all anyone will ever need! - Bill Gates
i think if i revisit it, i may at least for testing purposes make a new function that calls the KMP index function instead of trying to build it into the loop. that should make it easier to debug what's going on until i have the numbers right 06:57
Geth MoarVM/master: 5 commits pushed by (Samantha McVey)++ 07:23
samcv yay merges 07:24
07:50 geekosaur joined 08:11 zakharyas joined 08:24 robertle joined
jnthn morning o/ 08:48
timotimo morning jnthn 08:54
jnthn src/6model/reprs/MVMStaticFrameSpesh.c: In function ‘describe_refs’:
src/6model/reprs/MVMStaticFrameSpesh.c:124:5: warning: implicit declaration of function ‘MVM_spesh_arg_guard_gc_describe’ [-Wimplicit-function-declaration]
Seems new
timotimo oh, let me see 08:55
i wonder why i didn't see that
oh, it does output on my machine, so i was just negligient :|
Geth MoarVM: c78477cd0a | (Timo Paulssen)++ | src/spesh/arg_guard.h
put missing forward declaration of arg_guard_gc_describe in
 08:57
timotimo is reading the old heaptracker posts again 09:03
i don't think i quite grokked how exactly to make the heap snapshot contain entries that don't actually have a proper object to go with them, i.e. not sure how to properly replicate the effects of process_object; is it the "set_type_index" call that makes a difference? 09:07
jnthn What does "not a proper object" mean? 09:08
There are different kinds of entries
Some of them are synthetic things
timotimo yeah, i want some of those :)
09:08 zakharyas joined
timotimo the spesh arg guard doesn't have a repr for example 09:09
jnthn Ah, and you want something cleverer for it 09:10
I'd just add another synthetic 09:11
And add it to the enum in the viewer
timotimo OK, let's see about that
that's the big switch/case in process_workitem?
workitems* 09:12
jnthn Yeah
timotimo ah, set_ref_from is the magic thing that makes all of this work 09:24
oh, there's no api to put a COL_KIND-annotated thing into the heapsnapshot's work list? so there'd have to be an unbroken chain from the root-of-all-roots to every spesh arg guard if i want to use the special col kind? 09:44
except if get_workitem_idx learns how to spot a SpeshArgGuard 09:46
which it can't because no REPR or similar 09:47
i *can* have a special col kind for StaticFrameSpesh and put knowledge about its internals into heapsnapshot.c 09:48
i don't like this a lot :/ 09:53
jnthn tbh I'd just put Spesh Arg Gurd as the key and not try to represent the structure of which arg guard has what in the snapshot 09:57
timotimo oh, so it'd just directly hang off of the MVMStaticFrameSpesh 09:58
i think that's the way it is right now
i still haven't actually looked at a heap snapshot enough to *find* such an entry to see if it even works :D
haha, i want that "ignore inter-generational roots" patch now :D 10:04
gist.github.com/timo/800d375d4eeb0...87c98af185 - not terrible 10:05
nine: sorry, i hid the patch to have incidents away a bit. it now lives in the main repo in a branch "incidents" 10:08
AFK 10:15
brrt are spesh guard tree thingies ever modified? 10:16
i suppose they could be
jnthn A guard tree is immutable; updates copy/tweak/install 10:17
So that reads of it can be done lock free
brrt aha 10:18
well, i'm asking because i know you mentioned JITting the guard tree at some point 10:19
which i think should be doable
jnthn Guess we'd take the same approach
brrt and i was wondering what kind of lifecycle that'd have 10:20
jnthn Similar, I guess, though we might decide not to bother throwing away previous versions if they're just tacked on to the end of a candidate's JITting
brrt this reminds me of markmont's work. if we did have a open tmpflie handle that we would map, we could relatively easily map new pages out of that, and that might make the whole mmap/mprotect dance a little cheaper 10:21
oh, that is possible too, of course
we could add a section for it
and a label at the start of that 10:22
that would seem to be nearly optimal to me
10:45 ggoebel joined
timotimo don't we have to change a little bit of the tree rather than just tacking new parts onto the end? 10:46
or did you mean to just concatenate it to the existing page?
brrt no, you can concatenate however you compile the *current* tree to the last thing you specialize 11:04
so if i add a new specialization, i write a new tree, i compile that new tree after the JITcode for my last specialization, and then update the pointers to look at that 11:05
timotimo we can probably re-use parts from earlier that we can just jump to and get the right result from anyway 11:06
brrt either i'm not parsing you correctly, or it doesn't seem like a good idea 11:07
timotimo right, probably a premature optimization anyway
i can probably use atomics in the slowest part of the "calculate incident edges" code 11:11
Total objects: 555,773 11:16
Total references: 2,115,698
^- takes 10 seconds to calculate the reverses
11:28 Skarsnik joined
timotimo i do this in three steps: first i count up the number of incoming refs for every collectable, then i go through the list of collectables and give each a start and end point into a so-far empty references list, and finally i go through and put the from of every reference into the to's from-list 11:28
the last step is achieved using an array of "cursors", one for each collectable, that lets me increment after writing one entry 11:29
the count-up at the beginning takes 2.4s, the assignment one takes about 0.9, then 6.3s for the final assignment 11:32
i'd say the first part can also be done with two workers in parallel using atomic ops 11:33
Skarsnik timotimo, add stuff to query the profiler at run time or stuff like that x) 11:34
timotimo i assume that the access pattern for ++ing the "incomint edge count" will go sufficiently distinctly from the first and second half of the thingie
Skarsnik: the heap profiler you mean?
turning it on and off during a running program?
if you're willing to have the proram cooperate, you can do that with nqp ops :)
Skarsnik dunno, being able to take snapshot during an execution could be useful 11:35
timotimo you can literally nqp::mvmstartprofile(nqp::hash('kind', 'heap', 'path', '/tmp/myheapsnapshot')); nqp::mvmendprofile(); 11:38
since it forces a GC before it finishes up
Skarsnik ohhh
dogbert17 cool error message of the day: "MoarVM panic: native callback ran on thread unknown to MoarVM" 11:44
Skarsnik hm 11:45
dogbert17 it's from t/04-nativecall/21-callback-other-thread.t
Skarsnik actuallt that a good question. what happen to callback given that the C code can run in his own thread 11:46
timotimo yeah moarvm can't just run on a thread that doesn't have stuff set up for it
maybe in the future we could ask for another thread to spawn a threadcontext and such for us and wait for that to happen 11:47
Skarsnik hm 11:55
not sure how that work well if you write/read from stuff owned by the C thread 11:56
11:56 zakharyas joined 12:16 jdv79 left
jnthn dogbert17: Hm, odd...wonder how the thread is unknown to it 12:19
dogbert17 jnthn: yes, difficult to reproduce as well, grr 12:30
timotimo so, i was wondering if i should apply for a tpf grant for some perl6-related work i'd like to do 12:32
including web UI for the heap snapshot profiler, a new frontend (maybe TUI, maybe web ui) for the regular profiler, ... 12:33
the next call for applications is probably coming up very soon, it says "the first week of september" which has now passed 12:35
will have passed tomorrow*
lizmat timotimo: please go for it! 12:36
jnthn ++timotimo 12:37
timotimo oops, i did not expect actual enthusiasm like this :)
[Coke] timotimo: as the person who wrote that, I meant... the first ... full week? plus it's still the 7th, the first actual week isn't done yet. :P
timotimo right 12:38
[Coke] (it's coming very soon. :P)
brrt timotimo++ go do it
dogbert17 timotimo++ go go go 12:39
nwc10 timotimo: in theory you have a timezone advantage over TPF :-)
[Coke] nwc10: *thbbthp*
12:39 geekosaur joined
timotimo it'd probably be nice to have a single GUI that understands heap profiles, instrumented profiles, and a telemeh timeline, as well as make spesh and jit logs simpler to navigate 13:21
brrt instrumentation is good in general 13:22
timotimo by "instrumented profile" i just mean the first profiler we got 13:24
docs.google.com/document/d/102h_Nl...sp=sharing - here i've started my grant application text; could someone look at it and tell me if the tasks outlined in the synopsis are a reasonably small and achievable chunk of work? 13:46
brrt: ? :) 13:47
brrt oh, sure, but i'm not known for having my work reasonably small and achievable ? 13:49
timotimo heh.
brrt so i'm not the judge of that
timotimo i realize now that the heap snapshot profiler and the regular profiler are mutually exclusive, as you'd get loads and loads of profiler-related objects in the heap profiler 14:05
and the profiler also keeps some objects alive for longer than they would be normally
jnthn Yeah, I don't think you could sanely use the two together really 14:12
Not as they currently stand
Geth MoarVM: e86428d45e | (Jonathan Worthington)++ | src/spesh/inline.c
Don't try duplicate unreachable handlers on inline
 14:13
Skarsnik timotimo, maybe you can add in the goals : documentation/tutorial about that
timotimo Skarsnik: if i can up the requested amount by another 1k for documentation ... :D :D 14:15
jnthn timotimo: Looking quite good so far 14:16
timotimo How dare they ask for details and a proposed schedule 14:28
don't they know how the real world works :P
dogbert17 hmm, t/spec/S10-packages/precompilation.t fails on MoarVM HEAD 14:30
not ok 51 - precompiled module constants get updated on change
timotimo i might go for angular again for this web app 14:34
angular 4 this time; though angular 5 is supposedly coming out this month or something 14:36
good look, npm. getting an infinite loop of EACCESS for a folder it wasn't able to create for some reason 14:40
dogbert17 ==1977== Conditional jump or move depends on uninitialised value(s) 14:42
==1977== at 0x41960CD: MVM_string_gi_move_to (iter.h:68)
==1977== by 0x419A47B: MVM_string_index (ops.c:433)
==1977== by 0x41A225C: MVM_string_split (ops.c:1569)
==1977== by 0x40EB2F5: MVM_interp_run (interp.c:1637)
I get the above when I run './perl6-valgrind-m t/spec/S32-str/words.t' among others 14:51
the error seems to have been introduced with github.com/MoarVM/MoarVM/commit/a4...6715699f5c 15:01
timotimo i wonder if angular is a good fit at all 15:04
jnthn prefers react/redux fwiw 15:05
timotimo me, too. but the patent clause in their license is not cool
jnthn Oh?
jnthn didn't hear about that before
timotimo now i don't claim i understand it 15:06
github.com/facebook/react/blob/master/PATENTS
ilmari it basically means nobdy using react can sue facebook over _any_ patent 15:10
jnthn Well, they can, it just means their license to use react is terminated as a consequence 15:11
So they'd have to stop doing so
15:12 brrt joined
brrt timotimo, i like most of your proposal. 15:14
i'm wondering what you mean by 'framework for embedding domain knowledge'
also, i'm a tad bit worried about relying on the super adhoc nature of the JIT log
timotimo well, you know how i immediately spot BOOTSTRAP.nqp:1234 as "oh shit, we're doing multi resolution without the cache"?
brrt i did not know that 15:15
timotimo find_best_dispatchee is in there
brrt uhuh
timotimo there's also bind_one_param and ... i think just "bind", too
i think they both usually show up as "<anon>" or whatever; just the line number
for those the most important thing is to point the user at what exact piece of code causes many calls to these 15:16
but the user wouldn't know to even try, unless the tool somehow points them at it
jnthn Yeah, some "intelligence" to point out common pitfalls would be really neat where we can do it
timotimo i wonder how stable the line number of those functions are :D 15:18
brrt hehe 15:20
well, i see what you mean now
timotimo i think i'd prefer react/redux, too
brrt: also "hey these functions have BAIs in them for param_*_* ops, we can point the user at the signature and tell them what makes the jit unhappy 15:21
"
except of course if the jit fully implements all these param ops before the tool is finished :)
brrt well, yeah, stuff has to be a bit robust 15:22
or not
i don't know
timotimo i'm not sure what you mean 15:23
brrt i mean, for instance, that even-moar-jit dumps graphviz of expression trees in the jit log 15:24
timotimo right
brrt i mean, jnthn has done a pretty nice job of making the spesh log a consistent format
timotimo i don't find them very understandable, btw :S
brrt i have not done the same job with the jit log
timotimo right
brrt the graphviz dumps?
timotimo yeah, even after rendering them 15:25
brrt can't say i blame you
timotimo fwiw, making the jit log more useful can surely be part of the grant
brrt i do, but then agian, i'm not exactly the target audience
or i am
but my use is not super representative of perl6 developers
uhuh, but it does increase the scope tremendously
timotimo how, i just have to write a few printfs! :D :D 15:26
brrt :-)
timotimo fwiw, it'd be neat if i could include x86 disassembly for the really savvy users 15:27
maybe after the grant
brrt i'm wondering if there is a smarter way to do that than a directory dump 15:28
maybe writing a tarball
nine What's wrong with a directory dump? 15:32
timotimo immediately write the output of objdump rather than the raw bytes?
then it could go into thel og 15:33
brrt a directory dump is messy 15:37
and to identify the files, we need a separate file
not necessary if you write a single tarball, and you can use a tmpfile if needs be 15:38
so, for instance
you could setup the jit dump file to be a tempfile right before forking and execing moar 15:39
objdump and pipelining is less than awesome i think, but i could see ways to use it
timotimo set it to be a named pipe :P
brrt or that
timotimo the redux dev said in one of his talks that redux is also popular in the angular community 15:47
brrt what even is redux 15:49
timotimo here's a slide with 16 lines of code that are "basically all redux does" 15:50
do you know what react.js does?
nine no 15:52
brrt hacker news tells me it is evil facebook code
timotimo OK, so basically™ it lets you write render functions that turn a state (any object or object hierarchy) into some html stuff by plopping together components 15:53
then react takes your results, computes a diff for the html, and applies that
(or in the case of react native i guess there is no html involved or something?)
redux is used to handle the state stuff
jnthn Basically they are "let's do FP" instead of screwing around with a DOM :)
timotimo quite. 15:54
jnthn Which I found quite pleasant
timotimo redux asks you to express user interaction (and outside influences like results form get requests or websockets or what have you) as simple "action" objects, which are suppose to be simple javascript objects, i.e. what you could dump as json
you write a reducer function that takes a state and an action and transforms the state accordingly, and redux asks you to not modify the previous state 15:55
jnthn heads home; bbl
timotimo and all of these things can be composed nicely
the first talk i saw also had a nice time-traveling debugger thingie where you can turn on and off actions from "the past" and it'll give you a proper result, and also hot load your code and re-apply all the actions 15:56
nine Reminds me of the JS framework I created for work a couple years ago for a very dynamic user interface. That is centered around a global data structure representing the app's state. The UI can register event handlers for this data structure. 16:04
State changes have to be expressed in terms of create/update/delete events on the data structure. The UI reacts to those and only to those. User events must only trigger data events (those create/update/inserts) and the UI must only react directly to data events. 16:05
Sounds restrictive at first but this lets us store all events in a log in the data base. Re-loading a page just replays the existing events. And, since it's all basic create/update/delete any language that can read JSON (i.e. all of them) can recreate the data structure from the event log. 16:06
This includes PostgreSQL itself. So we have database triggers that react to events and update the server side state.
All in about 500 LOC including UI transactions and offline support, so it will send the events to the server whenever there's a network connection. 16:09
timotimo cool
16:13 ugexe joined
nine It was very much inspired by masak++'s talk at a twin city Perl workshop ages ago :) 16:14
timotimo is there a recording online? 16:15
nine I don't think so :/ Anyway, there's still this: masak.org/carl/ddd-cqrs-wiki/faq#co...egregation 16:18
16:34 dogbert2 joined 16:37 robertle joined 16:46 leont joined
nine Turning nativecallinvokejit into a real sounds easy at first. The real trickyness is finding out in which layer to add the support. The arg handling ops do not even exist in QAST. Call handling seems to be split between QASTOperationsMAST and mast/compiler.c 17:02
Skarsnik nine, you just jit the whole process to create the c call? 17:33
timotimo well, most of it 17:35
17:40 zakharyas joined
timotimo redux-saga seems neat 18:01
Skarsnik sound like an anime title 18:07
jnthn nine: I'd probably figure out a way to tweak it into github.com/MoarVM/MoarVM/blob/mast...s.nqp#L540 18:26
nine: Since that code path will handle the whole callsite business and spitting out the prepargs etc.
19:08 zakharyas joined
timotimo with non-globally-unique CUIDs the inline debug output will sometimes not be very helpful :) 19:30
19:33 AlexDaniel joined 19:35 coverable6 joined
timotimo especially with so many routines having "" as their name :) 19:36
19:37 bloatable6 joined
timotimo oh damn, these filenames are loooooong 19:46
ah, it's the path to the bytecode file, not the filename of the source 19:49
i wonder if it'd be nice to have the filename and line number from the most recent annotation as well 19:55
MasterDuke timotimo++ for grant work 20:06
22:19 geekosaur joined 22:34 mtj_ joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!