00:07 ggoebel joined 02:57 vendethiel joined 03:46 FROGGS__ joined 04:16 njmurphy joined 05:25 vendethiel joined 06:41 FROGGS[mobile] joined 07:02 flaviusb joined 08:17 vendethiel joined 08:51 vendethiel joined 09:17 FROGGS[mobile]2 joined 09:38 vendethiel joined
jnthn is part way to a solution that gets rid of the "timers etc. drive a core up to 100%" issue 09:39
lizmat jnthn++ 09:49
jnthn lizmat: Not sure immediately... 09:53
lizmat: It probably shouldn't make a difference.
lizmat pasting the question from #perl6 for posterity 09:54
[11:48:44] <lizmat>reverting 24aca6a8, seeing if that makes a difference
[11:53:01] <lizmat>it does... :( but why?
10:04 vendethiel joined
jnthn argh 10:41
Lazy deserialization contains a nasty race, it seems 10:42
This explains...various things.
lizmat indeed.. :) 10:43
jnthn Including the method cache thing.
lizmat even in one thread ?
jnthn No, turns out that the fact we SEGV in the method cache lookup is coincidence in the single-thread bug we saw. 10:44
That's an unrelated pre-comp issue so far as I can tell. 10:45
Anyway, I think my changes to avoid the "swallow a whole core" are OK. Not quite as good as I wanted, but seem to help a good bit 10:46
In theory, we can use prepare/check handlers to block/unblock ourselves for GC 10:47
In practice something didn't quite work out there, but maybe I'll try again after fixing the lazy deserialization stuff 10:48
That probably explains much of the 20% failure rate of TimToady++'s crashing Proc::Async thing 10:49
Ugh 10:50
lizmat is there a way to deserialize everything immediately, to allow tracing these types of issues ?
jnthn Actually we seem to hit the darn lazy deserialize thing almost all the time in my async socket test case now, I guess because it doesn't drive a core up to 100% and so we race harder on the rest. :/
lizmat: There's a #define you can set 10:51
Note that lazy deserialization hasn't typically been to blame for much so far as pre-comp bugs, though.
10:53 vendethiel joined
jnthn tries the good way of integrating GC with the event loop again, now he knows lazy deserialization is likely to blame for the crashes he saw 10:57
Yes, it works nicely 11:05
I need to tidy up the patch with some eror checking, but otherwise it's good and we'll hvae that "oh noes 100% CPU" problem solved. 11:07
lizmat jnthn++ # can't be said enough :-) 11:08
jnthn Plus diagnozed the lazy des race, so now need to ponder a solution 11:09
But for now, lunch :)
11:20 colomon joined 11:21 vendethiel joined 11:43 FROGGS[mobile] joined 11:47 Ven joined 13:32 ggoebel joined 13:45 FROGGS[mobile] joined 14:27 colomon joined
nwc10 jnthn: I'm wondering/hoping if the bug in lazy serialisation explans all that strangeness I found but couldn't (yet) track down when I tried to tweak how string indexes were serialised 14:47
14:50 colomon joined 15:03 colomon joined
timotimo i'm glad to see this figured out, as well as the 100% usage thing 15:15
15:26 vendethiel joined 15:47 Ven joined
dalek arVM: 58226af | jnthn++ | src/ (2 files):
Eliminate 100% core use for async things.

Use async wake-ups instead of an idle handler. For this to work out we also need a semaphore to avoid a race in sending the first task to the event loop. GC is also far better handled, using prepare/check handles so another thread can work-steal the event loop thread for GC purposes when it is sleeping awaiting an I/O notification etc.		 16:36
timotimo oooh jnthn is back :)
huh? why would we want to have other threads work-steal if the event loop thread is waiting for an I/O notification? 16:37
jnthn timotimo: 'cus the wait is deep in libuv somewhere
timotimo oh!
OK
jnthn Also, in most real world scenarios I've seen, we do have more threads than cores, so having some work stealing going on probably helps work against over-subscription duing the parallel GC. 16:38
Uh, we do *tend to*.
Not universally true of course.
Note that while this is an improvement overall, it *will* make things potentially seem worse, 'cus the lazy deserialization bug can be a good bit more noticable for some workloads now. 16:39
All being well, I'll dream of a good solution to that tonight and have it fixed tomorrow. :)
timotimo mhm
jnthn But anyway, our GTK apps that happen to use time won't drive a core up to 100% now :) 16:40
timotimo yeah, and sufficient amounts of other things
do we want to put a "use I/O directly rather than through libuv for synchronous I/O" thing onto moarvm's roadmap? 16:41
nwc10 sideways from that question, something that might be a win (on Win32) is not using libuv for stat. Or, more specifically, not doing a full "real" stat for anything that doesn't need all the fields of stat. 16:42
it's something that Perl 5 should be fixed to avoid, but no-one on Win32 seems to hear me suggesting it.
specifically, "file size" and "file permissions" and even "file type" don't require faking up a device and inode 16:43
and all the other stuff that is expensive to emulate a stat structure 100%
but as best I can tell, libuv only has a "stat" interface. which means faking stuff up, only to discard it 16:44
jnthn timotimo: Probably yes.
Anyways, dinner &
timotimo i think i'll write up a quick piece of text 16:51
since you manually update the website anyway, no need for a pull request for this
JimmyZ I have someting to say about libuv on win32, we have to do unicode -> utf8 -> unicode on win32, because libuv api args is utf8. 16:58
:)
sometimes
timotimo oh 17:00
dalek href="https://moarvm.org:">moarvm.org: 0ac79d2 | timo++ | roadmap.html:
roadmap: avoid libuv for synchronous I/O
 17:03
17:03 zakharyas joined
JimmyZ anyway, to be CJK-friendly, I prefer to use the unicode API on win32 17:03
17:16 Ven joined 17:43 FROGGS joined
japhb JimmyZ: What encoding is the 'unicode API' using? UCS-2? UTF-16? 17:48
FROGGS segfault in $string.NFD: rt.perl.org/Ticket/Display.html?id=125248 18:02
nwc10 jnthn: ASAN doesn't seem to hate you, but 3 tests are hung, multi-threaded 18:32
Id Target Id Frame 18:33
4 Thread 0x7ffc38486700 (LWP 25337) 0x00007ffc3d9b45bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 3 Thread 0x7ffc36f7b700 (LWP 25338) 0x00007ffc3e63b4d0 in AO_nop_full ()
at 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/gcc/x86.h:38
2 Thread 0x7ffc35a60700 (LWP 25339) 0x00007ffc3d9b45bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
* 1 Thread 0x7ffc40153760 (LWP 25248) 0x00007ffc3d9b45bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
full backtrace of all 4 threads: paste.scsys.co.uk/482769 18:52
slightly different structure paste.scsys.co.uk/482773 19:01
had 4 local patches atop MoarVM master. Which should not affect async. But retrying with the real master 19:08
(including having the FSA disableD)
19:45 Ven joined
nwc10 jnthn: now on master 19:46
paste.scsys.co.uk/482784
paste.scsys.co.uk/482785 19:48
paste.scsys.co.uk/482787 19:49
same three hangs as last time
and 5 other things fail. detail for 2: paste.scsys.co.uk/482790 19:53
linux hates everyone
20:41 Peter_R joined 21:29 colomon joined 22:40 vendethiel joined 23:16 vendethiel joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!