00:35 colomon joined
timotimo i'm not 100% sure where a dispatcher would come from given an inlined spesh graph 01:19
because there's a takedispatcher in each of the inlined BB's (single BB's, which is pretty good IMO) and the only thing i can see it do is reset cur_dispatcher to be VMNull
the graph it's inlined to doesn't do any *dispatcher things, though ... the only thing i can think of is that it comes from an onlystar proto perhaps? 01:20
02:28 colomon_ joined 02:32 colomon_ joined 03:45 FROGGS_ joined
timotimo i'm having trouble sleeping, it seems ... and thoughts about how stuff i've wanted in moar could work have been swirling around in my head 05:01
in particular, right now i'm thinking about what i'd call "deopt bridges" 05:02
which would be a bit of code we could generate that a deopt would jump to before doing the actual frame deopting/uninlining
for example: we've still got a const_s "&infix:<==>" in our code even though we inlined the call to that code, because if we need to deopt, we require that string to be in that particular register 05:03
so we could detect that a particular guard is keeping some registers alive and move that const_s into the "deopt bridge", then change the guard into a guard variant that jumps to a BB instead of deopting outright and then we add (or move?) an identical deopt annotation to the end of the bridge so that the deopt target bytecode address is correct 05:04
though it occurs to me that maybe instead of duplicating and switching the instructions for guards, we may want to build a deopt annotation that jumps to a BB instead of doing the deopt 05:05
dalek href="https://moarvm.org:">moarvm.org: b1da568 | timotimo++ | roadmap.html:
[roadmap] tick off the repr op devirtualization task
 05:09
05:13 jepeway joined
timotimo a reason we may not want to just paste the contents of inlined BBs into their caller's bodies is that they have an "inlined" flag ... but perhaps we only need that flag because we don't fix up enough of our helper data structures yet to keep the dead bb elimination from throwing them out wrongfully? 05:14
dalek arVM: c76197c | timotimo++ | src/spesh/dump.c:
display "Inlined" flag in the spesh log
 05:16
timotimo does something depend on the dominance children staying the same during a run through the optimization? 05:17
it's a bit upsetting to see &prefix:<++> be inlined by neither the static optimizer nor spesh ... 05:51
add insult to injury, a sink check is emitted for "++$pos"
TimToady hmm, usually it's $pos++ that needs the sink check, to turn it into ++$pos 06:18
07:02 zakharyas joined
dalek arVM: 1882d3e | FROGGS++ | src/6model/reprs/NFA.c:
fix thinko in NFA_EDGE_CODEPOINT_M_NEG, skids++
 07:43
07:48 jepeway joined 08:05 vendethiel joined 08:27 brrt joined
brrt \o 08:27
timotimo++ i think that is an awesome idea
second point - if you're sure it's not caffeine related, that insomnia of yours may become problematic in due course, and if it *is* caffeine related, well... 08:28
third point, i just got mail my grant app has been accepted :-)
but yeah, a deopt bridge would be awesome 08:29
not entirely sure how to do it yet
jnthn brrt: YAY! 08:30
brrt yes, that is what i thought too :-D 08:32
i have to run an errand, bbiab
dalek arVM: ea10ed6 | jnthn++ | src/6model/reprs/MVMMultiCache.c:
Add a missing barrier.
 09:01
arVM: 0334215 | jnthn++ | src/core/frame.c:
Fix a typo.
masak brrt++ # grant app accepted \o/ 09:11
09:11 brrt joined
brrt back 09:12
09:14 FROGGS[mobile] joined
FROGGS[mobile] \o/ 09:14
dalek arVM: 673b113 | jnthn++ | src/core/frame.c:
Tweak error reporting of wrong outer frame.

REPR will always be MVMStaticFrame, but it's useful to include the compilation unit unique ID.		 09:20
arVM: 10d00bb | jnthn++ | src/core/frame.c:
Include cuuid of invokee in error also.
 09:47
jnthn I have weak evidence that our frame pool is at the heart of various threading issues.
At least, disabling it makes an async echo server run and run, rather than explode after a while. 09:48
nwc10 there must be a joke something like:
FROGGS ohh, a sigspace joke :o), gotcha 09:49
nwc10 ABBA: "There is nothing we can do"
ABA: maybe this
jnthn I'm not sure if this one is ABA :)
nwc10 sorry for the pause - Unicode wasn't helping me
jnthn But yeah, with the frame cache disabled a whole load of different failures go away 09:50
FROGGS what's aba? 09:51
nwc10 something I only learned about last week: en.wikipedia.org/wiki/ABA_problem
FROGGS ahh, interesting 09:53
it is all about assumption, isnt it?
just told a colleague yesterday that believing that a condition must be true (in a certain SAP report generating script) is not a good way to debug something 09:55
jnthn With the frame cache it would always bomb with a variety of exceptions after around 5,000-25,000 requests
FROGGS "I don't know what is going on, but I don't need to check that because that condition cannot be false"
jnthn With it disabled I've had it make it well over 100,000 twice in a row and still running.
nwc10 FROGGS: I thought it was more about Ascension today. Assumption is 15th August (and a Saturday, so not a useful holiday) :-)
FROGGS :o) 09:56
brrt that is a pretty interesting result jnthn :-) 10:04
jnthn Doesn't seem to matter whether I disable placing in the pool or taking out of the pool, either fixes it. 10:05
The handling of the pool decidedly isn't thread safe, but otoh it should only ever be accessed by a single thread. 10:06
brrt now there's an assumption 10:07
jnthn Indeed. 10:08
FROGGS ahh, now I know what I can do until jnthn got enough of concurrency... I wanted to add tests for LTM 10:11
jnthn Aha...I may have figured what's going on. 10:17
FROGGS ohh, me is curious 10:18
10:18 vendethiel joined
FROGGS loves jnthn++'s ommit messages 10:18
commit*
jnthn In summary: when we do GC, we only stop the world for part of the time (that is, until we agree we saw all of the living things); then we unblock things and threads are free to do the final bits of the GC themselves (finalizing the collected objects) since they may have very different amounts to do at this stage 10:19
brrt when you create a new bug in a commit, it could be called an 'omit' message
jnthn If thread X is blocked during GC due to waiting on IO, a lock, etc. then work stealing happens 10:20
The thread that work-stole is responsible for GCing the threads it stole
The assumption about the frame pool gets broken I think because in some cases, a thread stops blocking on I/O and starts executing, but in another thread we're sweeping its nursery, and collecting closures, and since we "impersonate" the thread context we're collecting on behalf of then we end up with concurrent mutation of the frame pool 10:22
And thus corruption of it.
brrt that seems.. plausible 10:24
nwc10 I'm a bit confused as to why this only affects the frame pool. Why other stuff is robust against/unaffected by this class of unlawful concurrency
brrt can you force the work-stolen thread to block until gc is done 10:25
jnthn nwc10: Largely because many other places where the concurrency can crop up have been designed to cope with it.
nwc10: The frame pool is held per thread 10:26
brrt lunch &
jnthn nwc10: I can't think of a single other case anywhere that has an object we finalize hold per-thread resources.
nwc10 OK 10:27
jnthn The frame pool is decidedly an optimization
Unfortunately, this time it got caught cheating. 10:28
jnthn hasn't look all that closely at the frame pooling before 10:37
(Didn't impl it)
It doesn't acocunt for if a frame is hot, and so everything we ever invoke - even if it's only run once ever - gets a remembered pool entry 10:38
Turning it off and doing a "loop { }" shows it accounts for over 1.4MB worth of memory in a bsae Perl 6 process 10:39
jnthn does some timing measurements to see what it *wins* 10:41
Because it was written before we had our own fixed size allocator
timotimo cool, i like this news of bug finding
jnthn uh 10:42
It wins...nothing by my measurements.
o.O 10:43
timotimo huh, that's ... wow
jnthn I can imagine in more heavily threaded code it'd win more
Except our heavily threaded code is busted 'cus of it.
timotimo brrt++ # grant proposal accepted \o/
jnthn timotimo: You got a moment to test what I'm seeing also? 10:44
timotimo OK, please instruct me 10:46
jnthn OK
disable spesh inline for these so we actually measure invocation time
I'm trying with "sub foo() { sin(1e0) }; for ^10000000 { foo() }"
Which is just loads of calls
Tiem it 10:47
And then apply this patch to Moar, and time it again:
gist.github.com/jnthn/219169639acd3cb91890
timotimo that patch improves performance 10:51
by a very tiny amount
jnthn Uh, yes, I was thinking "noise" but all my measurements from it were a tiny bit below those I took beforehand.
timotimo i made many measurements; most with the patch were below 4.00 user and most without it were above 4.00 user 10:52
jnthn Wowser. 10:53
timotimo and then there's almost a megabyte saved in maxrss 10:54
jnthn So I think we may have an easier option than "fix it"
;)
FROGGS "rip it"
timotimo rip in peace, frame-pool
jnthn To be fair, it probably *did* help a lot when it was put in and we used malloc
timotimo maybe at some point in the future someone'll dig it back up, figure out why it doesn't help performance, and re-enable it with a fix 10:55
ah, that could very well be
another musing i had while between awake and asleep was this: since we know inlined pieces of spesh graph are "mutually exclusive", so are their registers. in theory, the registers could be re-used, so that the frame doesn't grow to the sum of all registers used, but instead the number of registers of the inliner plus the number of registers of the biggest inlinee 10:56
but then i noticed registers are typed, so they can't just be re-used without care
and that means there'd have to be some kind of register allocator, but also more work done when uninlining 10:57
i haven't actually looked at how uninlining works, so i don't know if the same approach with the deopt bridge code-gen would help with that exact situation
jnthn It'd be tricky 'cus of the uninlining, yes 10:58
timotimo our deopt is super cheap, but it costs us in other places
are you +1 on introducing a facts flag that points out if a register had its "users" counter "inflated" because of deopt? 11:01
jnthn What will we use it for? 11:03
timotimo i don't have a design yet, but i thought it would/could help with the deopt bridge
jnthn OK, I didn't understand that quite yet ;)
timotimo can you shoot questions my way to help figure it out?
jnthn Not while I'm in the middle of cleaning up the frame pool :P 11:05
timotimo of course
jnthn But yeah, I was seeing the "invoking a null object", "can't flatten null string" and, most often, "wrong outer" errors with the IO::Socket::Async ping server before now. 11:08
And it seems to run rather better at this point.
FROGGS that sounds awesome
jnthn It's leaking madly, but I know the async task pool leaks.
It's the same place I need to look at to fix the "pushes CPU load up to a full core" bug 11:09
timotimo at the moment i have a usleep(50) in that tight code that makes the cpu usage pretty much perfect
as in: almost 0 if moar's not doing anything
i realize it's not what we want, of course 11:15
jnthn *nod*
Yeah, the thing we have now is "I want something that works at all so we can explore the Perl 6 API for these things" 11:16
timotimo and that flag for the facts will at least show us - in the spesh dump - what registers have possibly been kept alive because of deopt safeness
these things mean the async/tasks/eventloop things?
jnthn Right
timotimo mhm 11:17
jnthn Well, Supply and so on.
We came out ahead overall, 'cus TimToady++ could get on with getting things named decently, and lizmat++ implemented us loads of extra functionality and tests.
timotimo writes a bit of code to show flags as a bunch of names on top of the number
jnthn But it still left a good bit to clear up. 11:18
timotimo because /me isn't very good at reading bitmasks as numbers in decimal
should a coderef argument be dumped as the cuuid + filename + linenumber of the ref'd MVMCode object and perhaps also that of the ->outer? 11:19
jnthn Not sure about the outer bit, but the other bits may well be useful 11:25
timotimo i'll probably just note if it has an outer or not
dalek arVM: 46899d2 | jnthn++ | src/core/ (5 files):
Remove the frame pool.

Before we had a fixed size allocator (and so malloc'd frames), it was a good win. These days, it seems to have no measurable effect on speed but costs around 1.4MB of memory for Rakudo. More seriously, however, it was vulnerable when the GC was freeing up closures in a thread it stole work from (because it used to be blocked), and that thread then woke up and started executing. This could have been fixed, but given the current frame cache design not seeming to help a great deal anyway the easier fix is to remove it.
11:25 brrt joined
timotimo sweet 11:26
nwc10 5 files changed, 19 insertions(+), 139 deletions(-) 11:27
timotimo the best code is removed code, right?
r29(7): usages=4, flags=269 KnTyp Dcntd Concr LogGd 11:29
i find this helpful
nwc10 the bugs aren't in the code that isn't there
jnthn OK, so now I've got the IO::Socket::Async ping server to survive into the hundreds of thousands of requests with a single thread hammering it, time to work out why it gets upset when two request threads hammer it. :) 11:30
Well, actually I think lunch first...
timotimo bon appetit :) 11:31
(om nom bugs)
dalek arVM: 01ecd8a | Timo++ | src/spesh/dump.c:
spesh dump shows little words for fact flags
 11:32
timotimo oh 11:33
mind if i re-push that commit? the author mail is wrong
dalek arVM: 3bed66e | timotimo++ | src/spesh/dump.c:
spesh dump shows little words for fact flags
 11:35
timotimo does it; it was quick enough i bet
vendethiel brrt++ # grats for the grant!
timotimo \o/
sorry for the rude push 11:43
11:47 travis-ci joined
travis-ci MoarVM build errored. Timo 'spesh dump shows little words for fact flags' 11:47
travis-ci.org/MoarVM/MoarVM/builds/62536504 github.com/MoarVM/MoarVM/compare/4...ecd8a8196f
11:47 travis-ci left
timotimo d'oh? 11:47
yeah, travis got rude'd 11:48
FROGGS aye
timotimo src/spesh/dump.c:231:46: warning: trigraph ??) ignored, use -trigraphs to enable [-Wtrigraphs] 11:50
append(ds, "???)");
why does this feel like a "blast from the past" to me?
getcode r1(1), coderef(gen/moar/stage2/NQPCORE.setting:92) 11:54
takeclosure r1(2), r1(1) 11:55
dalek arVM: 8640ee1 | timotimo++ | src/spesh/dump.c:
speshdump coderefs as filename + line number and "(closure)"
 12:05
nebuchad` nwc10: thanks for the ABA wikipedia link, I finally ends in the Actor model page and rediscover what I liked in Erlang ;-) 12:06
vendethiel if one wanted to figure out how moarvm works, should nqp be the starting point? 12:43
(some things are confusing, i.e. grepping for an opcode only finds matches in src/jit
)*
timotimo ah 12:44
nqp gives you an overview of how moar ticks with relatively little "magic" in the way 12:45
what exactly are you grepping for for the opcodes?
in interp.c they are "implemented" (most of them as calls to the implementing functions, actually)
and in there, it looks like "OP(abs_i)" 12:46
vendethiel timotimo: ah, I just looked at ops.h and grepped some of the #define s
timotimo ah
those are the generated ones :)
vendethiel ...I'll read the jit code later :P 12:47
timotimo hehehe
jit isn't necessarily the right place to start, i agree :)
anything in particular you'd like to hack on?
vendethiel not sure yet, probably get a feel on how it all works 12:48
timotimo OK
well, hit me up if you have any simple questions :P
but i'll be AFK for a bit first
also, are you interested in #perl6-gaming?
jnthn back 12:49
vendethiel I'll try again to get cairo working, I guess :-)
timotimo cool
i figured out i probably want to supply bindings to more SDL2 related libraries
like SDL2::gfx for some shape drawing and stuff
vendethiel I wrote a toy register-based vm (in c) for $school, I want to give a stack-based VM a try before dwelving deeper into "real" ones
.oO( a real virtual machine )
 12:50
timotimo moarvm is basically register-based
vendethiel that's how it looked in interp.c, at least 12:51
do we have tools to dump mast/qast once parsed? 12:52
timotimo for qast, you want to --target=ast and --target=optimize
for mast, you'd want to have a .moarvm file and moar --dump that
vendethiel (is that a human-readable format, or just moarvm bytecode?) 12:53
timotimo human-readable
vendethiel amazing, thanks
timotimo i mean --dump is human-readable, .moarvm obviously not :)
vendethiel yes, I guess the .moarvm files are the ones actually used
timotimo you already seem to know alot 12:54
vendethiel not really, know. don't worry, I'll ping you with my stupid questions :P
s/know/no/
timotimo OK. well you should get to know alot, alot is very nice
vendethiel alot++ 12:55
timotimo i like reading the spesh log output much more than i like reading moarvm dumps; they look similar, but the spesh log is in a control flow graph format and single static assignment
env MVM_SPESH_LOG=/dev/stdout perl6 -e 'say 1' gives you a whole crapton of output that you can use as an example to get a feel for it, if you want 12:57
vendethiel noted, thanks 12:59
FROGGS m: 'äaÄAÁbbBB' ~~ m:m/A+|bb|a/ # :o( 13:01
camelia rakudo-moar 2ea008: OUTPUT«(signal SEGV)»
dalek arVM: c65fd72 | FROGGS++ | src/6model/reprs/NFA. (2 files):
handle LTM for ignorecase+ignoremark
 13:02
13:02 brrt joined
arnsholt timotimo: Holy output batman! That's a lot of stuff =) 13:03
FROGGS ahh, now I know why it segfaults 13:13
dalek arVM: 751a1d9 | FROGGS++ | src/6model/reprs/NFA.h:
update NFA_EGDE_* constants
 13:20
vendethiel what's the 'a' block_id? seems like that's the only "validated" one (in, err, validate_block) 13:35
timotimo seems like some kind of "type" marker for different things 13:39
like, an 's' there would stand for a spesh op
vendethiel if (val->cur_mark && *(val->cur_mark) == 's') fail(val, MSG(val, "Illegal appearance of spesh op")); 13:42
right
why is *(val->cur_mark) sometimes used, and val->cur_mark[0] some other times? 13:43
FROGGS just because you can probably 13:44
jnthn grmbl. Seems removing the frame pool has exposed a bug that affects pre-comp 13:55
Could somebody else try t\spec\integration\precompiled.rakudo.moar with Moar HEAD to verify? 13:56
timotimo # When invoking cuid_3563_1431474887.30505 'STORE', provided outer frame 0x5139910 (cuid_6_1431611824.40465 '<load>') does not match expected static frame 0x16d8b60 (cuid_3686_1431474887.30505 '') 13:57
this?
jnthn Yeah, exact same, except addresses
timotimo sure
nwc10 paste.scsys.co.uk/478957 13:58
so yes, I also see that 13:59
jnthn yeah, and just backed up a few commits and it's gone
didn't re-comp
So it's something during loading
Most odd
timotimo i wonder if we were sometimes hitting this exact same bug before, but now we are able to catch it with a test file? 14:00
jnthn Very possibly
timotimo that's progress, then :)
jnthn I suspect the frame pool opt was hiding the bug.
Yeah
Seems I get all the fun today. Conc bugs and pre-comp bugs :P
Planning to work on the annoying one with the handles too 14:01
timotimo sounds satisfying :)
jnthn But, should un-bust this first given it blocks bumping MOAR_REVISION.
14:01 danaj joined
timotimo well, froggs already bumped moar_revision for the NFA changes up there 14:01
jnthn oops
Then everyone'll be seeing the issue in spectest 14:02
FROGGS :S
jnthn No biggy. I'll bump again once I figure it
It's a really odd one.
timotimo i'll be afk for a little bit and maybe then i can figure out the "deopt bridge" thing in better detail with you?
jnthn Maybe so, though I promised FROGGS some time to look at the serialization of the module DB today also... 14:03
timotimo right, that seems more useful at the moment :)
since froggs is so close to delivering something, as compared to me, who still has some design work to do
o/
[Coke] brrt++ # grant accepted. 14:04
FROGGS jnthn: btw, when is the best time for you to help me wrt serialization? 14:06
jnthn FROGGS: This evening maybe? 14:07
[Coke] (the best code is removed code, amiright?) --- I heard this in the voice of one of the guards in skyrim. Perhaps I've been playing too much. :)
FROGGS jnthn: that'd be lovely 14:08
jnthn OK, cool
aha 14:12
Seems to be a premature reclamation
nwc10 ASAN doesn't complain 14:13
was building to try with valgrind
jnthn It likely won't 14:14
However if you flip fixed size allocation into debug mode it probably might
nwc10 ah OK. I think I have the patch to do that in git stash somewhere 14:15
jnthn *nod* 14:17
It's certainly a deserialized context
14:24 brrt joined
dalek arVM: 27ee300 | jnthn++ | src/core/frame.c:
Fix ref-count management of deserialized contexts.

They should get a count purely out of the SC holding them, otherwise we can end up reclaiming them prematurely.		 14:27
nwc10 jnthn: shouldn't it be possible to spot that off-by-one error, if the refcnt dec code goes "-1? WTF! abort()" ? 14:32
jnthn nwc10: I don't think it woulda in this case 14:34
nwc10 OK
jnthn nwc10: The memory would get re-used for another frame and the count would be positive again
dalek arVM: df18912 | ven++ | src/core/validation.c:
Fix "instruction" typo in error message

Also replace a `*(val->cur_mark)` with `val->cur_mark[0]` for consistency.
arVM: bfa81c6 | FROGGS++ | src/core/validation.c:
Merge pull request #217 from vendethiel/patch-1

Fix "instruction" typo in error message
15:05 FROGGS[mobile] joined 15:25 FROGGS[mobile]2 joined
dalek arVM: ead50fd | jnthn++ | / (9 files):
Provide a "never repossess this" mechanism.

Used for things that we should never repossess. Will be used to avoid the bugs that result from PROCESS in Perl 6 getting repossessed.		 15:31
15:47 synbot6 joined 16:45 vendethiel joined 17:15 vendethiel joined 17:41 vendethiel joined 18:12 vendethiel joined
vendethiel jnthn:I have a question related to lexical scope and precompilation 18:58
how do you make, say, lexical scope of macros work correctly with precomp
module A: my $a; macro D($x) { $a = $x; }; # or something 18:59
jnthn Is it a different case from where one module gets a closure from another module and stashes it away somewhere? 19:02
masak don't think so. 19:27
jnthn Then it works 'cus you can serialize a closure taken during module B's compile time in module B's precomp blob, even if it's of a code object from module A 19:28
It's just another kind of cross-reference, like objects. 19:29
vendethiel that's interesting. I kinda want to read more about that 19:41
I wanted to write a toy language with macro, and separate parsing/codegen/vm, but couldn't figure out how to get macros into the mix :)
[Coke] I think masak already wrote that language. 19:43
vendethiel no, because masak wrote that language.
(yes, I put that dot to align it :P)
dalek arVM/jsoff: 4b48730 | FROGGS++ | src/core/interp.c:
add MVM_get_idx_of_sc to debug output
 20:01
arVM/jsoff: 7a34a45 | jnthn++ | src/core/interp.c:
Sketch code we need for allocated SC idx.

Should factor this "clear SC" logic out.		 20:19
arVM/jsoff: b4feb1a | jnthn++ | src/6model/sc.c:
Check owning object wasn't disclaimed.

Some objects are serialized as part of an enclosing object (more for historical reasons than good ones at this point, alas). Check we did not clear an SC when doing repossession.
japhb jnthn++ # concurrency fixes! \o/ 21:34
21:39 colomon_ joined 21:43 colomon joined 22:18 jepeway joined 23:53 vendethiel joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!