00:46 Ven`` joined
samcv Final grant report now published cry.nu/grant-report/ 00:50
01:55 ilbot3 joined 02:58 Ven`` joined
MasterDuke samcv++ 02:59
yoleaux 25 Sep 2017 09:56Z <cuonglm> MasterDuke: can you tell me where is the code for `-n` command line flags
25 Sep 2017 10:55Z <jnthn> MasterDuke: I'm surprised join doesn't already know that trick! Yes, let's do that, it's far more general
03:11 zakharyas joined 03:19 bisectable6 joined 03:43 Ven`` joined 04:07 coverable6 joined, committable6 joined, nativecallable6 joined 04:31 vendethiel- joined 04:32 geekosaur joined 05:37 domidumont joined 06:11 domidumont joined 06:50 brrt joined 07:01 leont joined
brrt good * #moarvm 07:03
lizmat brrt o/ 07:17
brrt ohai lizmat 07:19
thanks for the weekly :-)
lizmat you're welcome! 07:22
07:49 brrt joined
lizmat samcv++ : cry.nu/grant-report/ # dated 20 Sep, did I miss that for the P6W ??? :-( 08:25
nwc10 if we can have point releases for MoarVM, Rakudo etc, can we have point releases for the weekly? :-) 08:26
lizmat hehe... well, eh, at least now I don't have to worry what the main story is going to be next week 08:28
samcv lizmat, no 08:33
i released it today
lizmat ok, *phew* :-)
samcv i should change the time to be accurate 08:34
ok i think it's updated now
lizmat :-) 08:35
jnthn samcv++ # awesome work, great report :) 09:27
dogbert17 jnthn: did you sample the foodtrucks this weekend? 09:39
jnthn dogbert17: No. That would have been an excellent reason to have a stomach problem on the evening, but I ended up with it without them :P 09:40
Did make it to the national railway day event though
dogbert17 cool, was it a nice event? 09:41
jnthn Saw the stream train. It was absolutely packed out, though, so we figured we'd take a ride another time :)
But yeah, was nice :)
dogbert17 remembers wanting to be an engineer when he was a kid 09:44
let me know when you're well enough to tackle a mystery 09:47
"nasty" stuff like 'Internal error: zeroed target thread ID in work pass' 09:50
jnthn Hm, that's the kind that a MVM_GC_DEBUG=1 or 2 and a small nursery can often narrow 09:54
dogbert17 this happen in one of our favorite test files, albeit seldom, t/spec/S17-promise/nonblocking-await.t 09:56
jnthn Ah, is that the one that was also hanging for some folks? 09:57
dogbert17 yeah, want to check out a gist? 09:58
jnthn Sure 09:59
dogbert17 gist.github.com/dogbert17/ead3d659...9b12e41c0b
jnthn Ah, the hanging one was S17-supply/syntax.t 10:00
Is that with MVM_GC_DEBUG=1 or without?
dogbert17 that's the one making TimToady's life misearble :)
without
can try with it if you want 10:01
jnthn Yeah, please do
dogbert17 ok 10:02
jnthn Darn, it seems it was my supply-related changes that have regressed one of the cro-websocket tests
nwc10 it has hung for me too at times
jnthn wonders if this hang and the hang in the cro-websocket tests are related 10:03
ooh, I get it on my home machine 10:04
dogbert17 yay
nwc10 and at this point jnthn declares "lunchtime!" and exits stage left
jnthn Not feeling up to going to the office at least has some benefit :P
And yeah, it's nomming CPU and memory 10:05
dogbert17 being at home has its perks :)
jnthn Extracting the test from its test file also hangs 10:06
Anyway, that it works out on my fast machine and hangs reliably in my slower VM makes it likely it's a race 10:07
Hmm, where's that debug thing I made last week... 10:11
Well, it confirms that timer events file up 10:19
Doesn't say why
As in, doesn't say something else is holding the lock for a long time 10:20
dogbert17 I noticed something else yesterday, if I decrease the size of the nursery I can get a couple of lock related tests to hang. Nursery size like 32768 * 8 10:34
it would be a simple task to break into the hanging programs with gdb :) I did that yesterday with the annoying syntax.t test 10:35
wrt to the syntax.t test each time the program "hanged" and I used gdb one of the thread was always at this line 10:39
MVM_gc_root_add_frame_roots_to_worklist (tc=0xb26a6c08, worklist=0xa8ccf050, cur_frame=0x9d699e20) at src/gc/roots.c:366
but perhaps that is to be expected ? 10:40
timotimo "perf record" on the hanging test also says the time spent in gc_root_add_* stuff is a whole lot 10:44
dogbert17 timotimo: it looks like this in gdb, gist.github.com/dogbert17/ca7ac0e2...096eaa4beb 10:46
timotimo i'm not sure if there's any frame that has a whole lot of locals in it 10:47
or if there's just a whole lot of frames on the moarvm stack. which i don't think there would be
digital cookies for someone who builds an extension or userscript that lets me go from a line of code that contains a filename and line number (including path) directly to the right github repository and file in it 10:49
jnthn I think I've found a problem in the Async::Lock queue-on-recurse mechanism 10:57
timotimo cool!
jnthn Uh, Lock::Async
dogbert17 cooool :)
moritz jnthn: that sounds like the kind of thing where you don't want to have problems :-)
timotimo that could indeed be what makes the supply syntax test hang sometimes 11:00
jnthn It's possible, yeah
I think the issue is here: 11:03
elsif self!on-recursion-list() {
# Lock is already held on the stack, so we're recursing. Queue.
$try-acquire.then({
LEAVE self.unlock();
self!run-with-updated-recursion-list(&code);
});
}
It sticks a .then on the Promise that is kept when the async lock is available
timotimo there could already be thens on there? 11:05
jnthn Yeah, if two things want to queue up there, then it sticks both onto that Promise. The .thens will fire. Both now think they have the lock, run code as if they have it, and then both go to unlock it. 11:06
timotimo but shouldn't that throw the "can't unlock when not locked" error? or does something else quickly enough jump in and hold the lock?
jnthn Yeah, I'm curious why I don't see that 11:07
timotimo so every time you recurse you add another simultaneous track of pieces that think they can have the lock
jnthn Which makes we wonder if this really is the problem in question
timotimo you could try having an "id" for every locking phase
and make sure the locks and unlocks match up their ids
jnthn Well, the easy fix is just to .then and re-contend 11:10
Doesn't help, though :( 11:15
timotimo dang
jnthn Which, given we didn't see the expected double-unlock error, isn't entirely surprising
m: my $s = Supplier.new; react { whenever $s { .say }; whenever Promise.in(1) { $s.emit("a"); $s.emit("b"); $s.emit("c"); $s.done() } } 11:16
camelia a
b
c
jnthn Hm, I kinda expected that to trip over it
m: my $s = Supplier.new; react { whenever $s { .say sleep 1; say now; }; whenever Promise.in(1) { $s.emit("a"); $s.emit("b"); $s.emit("c"); $s.done() } } 11:17
camelia ===SORRY!=== Error while compiling <tmp>
Two terms in a row
at <tmp>:1
------> Supplier.new; react { whenever $s { .sayā sleep 1; say now; }; whenever Promise.i
expecting any of:
infix
infix stopper
ā€¦
jnthn m: my $s = Supplier.new; react { whenever $s { .say; sleep 1; say now; }; whenever Promise.in(1) { $s.emit("a"); $s.emit("b"); $s.emit("c"); $s.done() } }
camelia a
Instant:1506424678.071537
b
Instant:1506424679.075984
c
Instant:1506424680.077176
jnthn Hm, and I'd have expected that to have issues, but it works 11:18
m: my $s = Supplier.new; await start react { whenever $s { .say; sleep 1; say now; }; whenever Promise.in(1) { $s.emit("a"); $s.emit("b"); $s.emit("c"); $s.done() } } 11:19
camelia a
Instant:1506424799.608883
b
Instant:1506424800.611684
c
Instant:1506424801.612954
timotimo m: my $s = Supplier.new; await start react { whenever $s { .say; sleep 1; say now - $*INIT-INSTANT; }; whenever Promise.in(1) { $s.emit("a"); $s.emit("b"); $s.emit("c"); $s.done() } }
camelia a
2.12159250
b
3.1251021
c
4.1262031
jnthn The first line can also be my $s = Supply.interval(0.001).head(0) and it still hangs 11:27
Ah, though head is implemented using a supply block :)
So it's not actually a golfing 11:28
timotimo how come .head(0) actually does anything at all?
dogbert17 jnthn: what did you get for lunch? 11:50
dogbert17 hasn't eaten yet :( 11:51
jnthn Still eating simple stuff for today 11:54
Had some mashed potato earlier, will have some vegetable soup soon :) 11:55
11:55 brrt joined
jnthn Still no particularly good leads on the bug 11:55
Instrumented the supply block guts some, but they all look to be behaving well 11:56
11:57 patrickz joined 11:58 zakharyas joined
lizmat jnthn: would it make sense to have all BUILDALL (only) methods to share the same Any:D: signature ? 11:59
timotimo hm, not Mu? 12:00
lizmat no, because the default .new is in Any, not Mu 12:03
timotimo: so you're saying that would make sense ? 12:04
timotimo haven't spent enough thought on it
currently puzzling over the runaway quote thing
lizmat it would effectively bring the number of Signature objects for BUILDALL down N to 1 (like it is now) 12:05
jnthn Would probably work out 12:06
lizmat ok, lemme try :-)
jnthn It seems that we're never unlocking an async lock 12:10
Not clear where that's going wrong jsut yet
12:15 brrt1 joined 12:27 brrt joined
lizmat do we have anything like compile constants in nqp that I could let depend on an ENV variable at compile time ? 12:35
dogbert17 jnthn: finally managed to get a 'Adding pointer %p to past fromspace to GC worklist' when running t/spec/S17-promise/nonblocking-await.t. Gist gist.github.com/dogbert17/f8cb9974...da44ef40da 12:42
jnthn dogbert17: Ah, good that we can catch it under that 12:51
Though, hmm, that doesn't narrow it down a load
lizmat jnthn: it looks like auto-generating a BUILDALL is going to break some tests that assume BUILDALL lives in Any
timotimo yeah, frame roots :\
jnthn lizmat: Which tests? 12:52
lizmat S12-introspection/methods.t 1-3, 9, 15, 20, 22-24, 42, 46, 57 12:53
jnthn oh, interesting
m: class C { submethod m() { } }; say C.^methods 12:54
camelia (m)
lizmat ah, it should be a submethod is what you're saying :-)
jnthn Oh, I thought you'd done it that way already ;) 12:55
It should but it won't help :)
Since .^methods includes those
lizmat I guess we can simplify the BUILDALL to a generic one that just returns self if there are no elements in the BUILDPLAN 12:56
jnthn: not sure it would make sense to make it a submethod though 12:57
if a subclass only adds methods, shouldn't they be able to share the same BUILDPLAN ? 12:58
12:58 brrt joined
lizmat especially if it has a Any:D: invocant ? 12:58
s/BUILDPLAN/BUILDALL/ ?
jnthn What it the subclass uses a custom metaclass that doesn't trigger BUILDALL generation? 13:01
lizmat is that possible ?
jnthn Yes
I think submethod is safer
lizmat so all the "but" cases would just get a new BUILDALL method 13:02
dogbert17 reboot & 13:25
13:31 dogbert17 joined
timotimo brrt, zoffix found a change to the optimizer that causes a segfault, but only if the jit is enabled. somehow a null pointer gets into the first argument of the reprconv getarg function 13:41
hm, may be a difference between how the interp.c handles the object being null and how the reprconv does it 13:42
er, sorry, get_attr_o of course
the interp just does IS_CONCRETE, which also segfaults on a null pointer 13:43
oh
it's calling that on self, but this is a sub, not a method
dogbert17 this syntax.t code is interesting, if I change 'await do for ^4' to 'await do for ^2' the program consistently take 1s to execute, if I increase it to 'await do for ^3' is often takes 25s to execute but once in a blue moon this also takes 1s 13:46
dogbert17 (has four cores in his vm) 13:47
timotimo aye
i see something similar, getting that number down gets me to pretty much always succeed it
what happens on your machine if you give .interval a second argument, like 0.01?
dogbert17 will try
with 0.01, ^3 takes 1.7s and ^4 takes ~2s 13:49
timotimo but it always succeeds, right?
dogbert17 yep 13:50
even ^5 succeeds in short order
0.005 also works quite well, is there some queue which gets filled up somewhere? 13:51
jnthn So far I've traced it down to an await on the async lock never coming back
Even though the lock is released 13:52
By now am down in the awatier guts
[Coke] btw, I have some Proc::Async code that hangs if I start more than X promises simultaneously (on 2017.09 release) github.com/perl6/doc/blob/master/xt/aspell.t (check out docs, then TEST_THREADS=10 perl6 xt/aspell.t triggers it here. have to have aspell and awk installed. (need to get rid of the awk requirement there.) 13:54
dogbert17 strace just says futex(0xb4a3fbc, FUTEX_WAIT_PRIVATE, 1, NULL 13:57
[Coke] (looks like it works with 6, then hangs with 7). happy to provide more diagnostics if needed.
(ah, no, six can hang too, just not up front.) 13:59
Geth MoarVM: c40be90d36 | (Timo Paulssen)++ | src/jit/x64/emit.dasc
jit: getlex_o shall give VNMull rather than null

you can trigger that in nqp by using self in a sub that doesn't have a self available from outside.		 14:02
dogbert17 timotimo: when I run the syntax program with ^4 the program seems to start 8 threads, after a number of seconds that increases to 9 and an equal amount of time later it goes up to 10. A few seconds after that the code completes 14:31
jnthn [Coke]: The new scheduler was merged shortly after 2017.09, and may well resolve the issue 14:32
(Or at least, make it work for longer...eventually it's probably a job for 6.d.PREVIEW)
[Coke] jnthn: doh, forgot that came in after. updating! 14:38
jnthn It's looking like, after all my supply bug hunting, it may be a scheduler bug that I'm looking at 15:08
I eventually chased it down to the unlock promise not being scheduled
15:09 AlexDaniel joined
dogbert17 jnthn++, that was a tough one 15:09
jnthn And I think I can see what's going on, too
The timer thread is busy pushing new work 15:10
So we're eating resources
dogbert17 is that the 'memory leak'?
jnthn "Leak" is just ever-growing work queue
On my 12-core machine at the office, it happily spawns up to 12 threads before it starts thinking if it really should add any more
On this 4-core VM that point comes far sooner 15:11
dogbert17 so that's why it was difficult to reproduce at work
jnthn Right 15:12
Or just impossible
On that machine
:)
It's using blocking react in this test file
That's why this also goes away if the loop is for ^2 or something in that test also
Anyway, the blocking react really blocks a bunch of threads
The progress elsewhere confuses the schduler into thinking everything is looking great 15:13
*scheduler
dogbert17 so it does nothing?
jnthn So it doesn't add more general workers
Right
It should probably also look at how long since something in the queue made progress. 15:14
And use that as a "we might be deadlocked" heuristic also
dogbert17 sounds like a clever move 15:15
jnthn Yeah, want a break, then will look at that.
dogbert17 so the unluck promise wasn't being scheduled because there was no bvious reason to do so ? 15:16
*obvious
[Coke] jnthn: that upgrade definitely helped. (I should also be funneling all these promises through something that is concurrenet safe rather than an array) 15:29
15:39 AlexDaniel joined 15:42 brrt joined
jnthn dogbert17: yeah, pretty much that 15:48
16:01 robertle joined 16:23 leont joined
brrt hi #moarvm 16:26
jnthn o/ brrt 16:27
brrt why, and how, does my tag still show moarvm 2017.8 after merging with origin/master
\o jnthn
that's because that's what git describe is giving me 16:28
oh, because of rebasing 16:29
my bad
16:32 MasterDuke joined
MasterDuke jnthn: fyi, github.com/MoarVM/MoarVM/pull/705 now optimizes copying strands and when only one element is in the list to join. no hurry, just getting a ping in 16:37
yoleaux 15:46Z <AlexDaniel> MasterDuke: did we have a ticket for ļ½¢say ā€œfoo"; my $x = 42; say $xļ½£ ?
16:44 domidumont joined 17:13 dalek joined 17:19 robertle joined 17:42 SourceBaby joined 18:17 squashable6 joined 18:25 quotable6 joined, committable6 joined, unicodable6 joined, coverable6 joined, bisectable6 joined, bloatable6 joined, nativecallable6 joined, benchable6 joined, releasable6 joined, greppable6 joined, evalable6 joined, squashable6 joined, statisfiable6 joined 18:35 Ven`` joined 19:09 lizmat joined 19:34 TimToady joined 19:46 Ven`` joined
dogbert2 MoarVM Panic caught when running t/spec/S14-roles/mixin.t, gist.github.com/dogbert17/696587a8...9e9f01447e 19:48
timotimo oh, huh, the mixim cache again? 19:49
dogbert2 is there such a beast?
timotimo hm, maybe i'm thinking of the parameterization cache 19:51
my code to squish phi argument lists in spesh causes some kind of infinite loop or something in the very last step of rakudo installation %) 19:52
dogbert2 can you break into the process with gdb and see what's going on? 19:54
timotimo i could, but maybe i'll put the squishing into the output instead %) 19:57
it's not actually an optimization per se, as phi nodes don't appear in the generated code at all
dogbert2 the old println trick :)
timotimo well, how fun is it to look at this (screenshot upcoming) 19:58
i.imgur.com/9mqMta9.png 19:59
and this is a very tame case
i.imgur.com/Q3l5k16.png 20:02
dogbert2 looks moderately fun :)
timotimo hum. graph_spesh relies on a MAST/Ops.p6 module that apparently no longer gets written while building moar 20:13
the file i have locally is from november 2014
samcv good * 20:17
timotimo heyo samcv
samcv heyo
timotimo Ops.p6 doesn't show up in any commit diff of all of moarvm
ilmari not MAST/Ops.nqp? 20:22
timotimo nope, a p6version of that
um, huh 20:28
m( it's under tools/lib 20:29
so why is this sp_getlex_ins without operands in the graph drawing 20:31
oh, and an sp_decont likewise, weird 20:34
must be because the Ops.pm file was out of date i guess 20:36
hm, i put that file into gitignore
well, hardly anybody besides me uses that tool anyway, so ... :) 20:47
it also gets very confused by multiple annotations in a row 20:49
all in all, a grant well done 23:28
23:56 MasterDuke joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!