Parrot 4.7.0 "Hispaniolan" | parrot.org | Log: irclog/perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 27 August 2012.
00:16 whiteknight joined 00:20 kid51 joined
whiteknight good evening, #parrot 00:30
benabik o/ whiteknight 00:32
whiteknight hello benabik 00:33
dalek sella/commandline_2: 2476c29 | Whiteknight++ | src/commandline/ (2 files):
[CommandLine] Break ArgumentDef out into two subclasses, so we don't need to do an if/else dispatch on every single method call
 01:02
04:14 wagle joined 04:53 kurahaupo joined 05:23 benabik joined 05:55 wagle joined 07:37 kurahaupo joined, kurahaupo left 08:11 lucian joined 08:19 Psyche^ joined 09:49 schmooster joined 09:54 schmooster joined 10:26 woosley left 10:55 ligne joined 11:25 schm00ster joined 11:34 schmoo joined 11:52 JimmyZ joined
nemesys anyone else working on threads currently? 13:03
except nine
dngor Are you polling for a lock on that feature? :)
nemesys not really, no - rather trying to understand more of it 13:04
read nine's thesis about it, now trying to get the current picture
if I understood it correctly, parrot itself uses 2 threads anyways, the io_thread for dispatching signals and a "main" thread 13:06
the main thread than allows running more "tasks" 13:07
13:08 benabik joined
nemesys i'd rather say I do lack architecture understand of parrot 13:10
s/understand/understanding/
13:18 PacoAir joined 14:17 JimmyZ joined
JimmyZ nemesys: here are some posts about threads on parrot, FYI: whiteknight.github.com/allposts.html 14:18
14:40 bluescreen joined 14:42 nnunley joined 15:13 dmalcolm joined 15:33 JimmyZ joined 16:12 contingencyplan joined 16:55 whiteknight joined
dalek p/toqast: a94c294 | jnthn++ | src/NQP/Actions.pm:
Fix an $/ handling issue, which fixes two more of the Rakudo spectest file regressions.
 17:08
kudo/nqpqast: 44f06cf | jnthn++ | src/Perl6/Grammar.pm:
Update a workaround for a bug (that needs a proper fix).
 17:09
kudo/nqpqast: 6c8ef80 | jnthn++ | src/Perl6/Metamodel/AttributeContainer.pm:
Fix subtly buggy code that used to work for entirely the wrong reason. Deals with another regression.
17:09 jlaire joined
whiteknight good afternoon, #parrot 17:29
japhb (Carried over from #perl6): What's the current state of the thread project in Parrot? Is that now done, or "coming soon", or still a ways out? 17:45
whiteknight soming soon. I plan to merge it on tuesday 17:46
tadzik ooh yes
japhb Ah, has a date even, excellent.
whiteknight why, is rakudo itching to start parallelizing?
japhb We're itching for 6mojo, which means either good non-blocking IO or good threads. 17:47
whiteknight non-blocking IO will come after threads
japhb whiteknight, implemented in terms of threads, or using async primitives from the OS? 17:48
sri how expensive will threads be?
whiteknight japhb: I don't know. Threads gives us a toolset to handle asynchronous callbacks, so threads will be involved somehow 17:49
sri along the lines of jvm threads (1kb) or perl5 (1mb+)?
whiteknight doing asynch IO in a platform-neutral way without using threads too much may be a hassle
sri: for a common, single-threaded application there's negligible cost for adding it
sri: nine has measured some nice speedups for parallelizable tasks with threads 17:50
japhb whiteknight, I think sri was asking about per-thread overhead ...?
sri NIO in the jvm is not a bad platform independent way to abstract non-blocking I/O
whiteknight I don't know what per-thread overhead will be. Probably closer to the jvm version than the p5 version 17:51
japhb good to hear! :-)
sri right, i meant per-thread overhead
whiteknight Parrot uses a hybrid approach of tasks+threads, so you can schedule multiple tasks on a single thread for virtually zero overhead
it's like stackless threads in python
keep in mind that the initial version found in the merge may need to be tuned, optimized, etc. So things will get better if they aren't great initially 17:52
japhb nodnod
sri interesting 17:54
whiteknight sri: if threading is used, you'll get a pool of threads. You create tasks and the tasks are dispatched onto threads. In a 1:1 system, it could be relatively heavyweight, but in an n:1 system you get much lower overhead 17:55
especially since n tasks running on a single thread are able to share data without locks 17:56
so there's no risk of memory corruption
sri ah, tasks are pretty much coroutines? 17:59
or green threads 18:00
sri was hoping for real threads like in jruby and rubinius 18:02
lizmat sri: fwiw, the weight of an Perl 5 ithread depends on the amount of code compiler / number of modules loaded 18:06
as *everytihing* (apart from optrees) is copied to a new thread when a thread is started
sri which makes them mostly useless 18:10
the ruby folks have been doing some pretty exciting stuff with their cheap real threads celluloid.io
whiteknight sri: yes, tasks are green threads 18:15
sri: but they get dispatched to real threads
But the system is much more flexible than saying every task must be a thread
sri and each real thread contains a whole interpreter?
whiteknight yes, believe so
sri :(
whiteknight although it doesn't get a copy of all the interp data 18:16
sri tbh. that kills concurrency in perl6 for me
lizmat sri: what does?
whiteknight and what would you prefer it do? 18:17
keep in mind, this is only the first go at it
well, that's not quite right. It's a flexible system that we can definitely build on and modify
What we need most are real use-cases from real users 18:18
dalek kudo/nqpqast: f962cf8 | jnthn++ | src/Perl6/Grammar.pm:
Fix $< parsing (just replace what we had before with what STD has). Fixes the S05 failures.
sri jruby threads for example are not isolated, if you modify a data structure from multiple threads you're responsible for locking
whiteknight so, what does that buy you? That pushes consistency requirements onto the programmer, but doesn't make them go away 18:19
sri real concurrent threads that are cheap 18:20
yes, you can use actors for that
see Akka for example
whiteknight sure the threads are cheap, and all the locks and deadlock potential and spinwaiting are expensive
it just pushes the costs onto the programmer
sri right, but it makes you much more flexible 18:21
whiteknight What do you mean?
sri akka.io
whiteknight The hybrid task/thread model we're preparing to merge was very much inspired by actors and concurrency models like python stackless and erlang 18:22
I'll keep saying that it's not perfect in the first version, but much of the potential is there to do more, more efficiently 18:23
sri perhaps i'm not understanding how task scheduling works then
whiteknight A task is basically just a continuation: small and cheap to invoke. Tasks are queued to threads, where they are run
create tasks, schedule them to threads. 18:24
if you have multiple tasks on a single thread, they preempt each other
or, they can
if you're on a single-threaded platform, this system degrades very gracefully with no changes needed to most programs 18:25
It also really doesn't have locks, because data shared across threads is basically read-only
sri you're basically back to non-blocking I/O if you want to scale a server to 10k active sockets 18:26
whiteknight what do you mean?
sri i just don't see scheduling of 10k green threads working where each is watching a socket 18:27
whiteknight having 10k active sockets in almost any configuration is going to be a problem
sri perl5 does it without problems on my old macbook :) 18:28
(non-blocking I/O of course)
whiteknight how do you have perl5 doing it, with 10k ithreads, each with a socket to watch?
sri that would be impossible i imagine :) 18:29
whiteknight right, you wouldn't do it that way in parrot either
sri but you do it that way in jruby
or rather... can do it that way
or Java/Scala 18:30
whiteknight in parrot you could easily attach all your sockets to a Select, then use that to dispatch short-lived tasks to handle incoming requests
you wouldn't want to create 10k tasks up front and have them all busy-wait
since tasks are just continuations and they have a very small memory footprint, you can create them quick and maintain a large number of them in a queue somewhere 18:32
I don't remember the exact numbers, but nine did benchmarks with 10k tasks or more running in a single process
sri i guess you can already have that in perl5 with ithreads and Coro 18:33
whiteknight sri: If you have some specific ideas you want us to consider, I'd definitely like to hear all about them 18:35
our system will get better with feedback
sri guess i was hoping for something that doesn't make huge sacrifices to prevent data corruption 18:39
whiteknight I wouldn't call too much of what we've done "huge sacrifices" 18:40
sri yielding between coroutines can be a huge pita
whiteknight it's just a continuation invoke, no more expensive (theoretically) than a method call/return
of course, method call/return in our system is a little more expensive than it needs to be
sri real threads could just use blocking syscalls and not worry about yielding 18:41
whiteknight Yeah, and you can use real threads and blocking syscalls
or, you can use a blocking syscall and preempt a new green thread to run during it
sri but that would be expensive again :)
whiteknight I think you and I have very different definitions of "expensive" 18:42
sri cheap = in the 1-2kb overhead range
Coro in perl5 is 4kb i believe per coroutine
whiteknight I don't know how big a parrot task is, but I doubt it's 4k 18:43
I would estimate it's 1k or less per task 18:44
per thread interp memory usage is higher because it allocates thread-local pools
so that's hard to measure what is actually being used
sri i get where you're coming from, but making sure that green threads don't block each other might be just as annoying as corrupted data structures (if you forget locking) 18:45
you basically make scheduling green threads your users problem ;p 18:47
it's either... schedule your tasks... or protect your data 18:48
whiteknight Green threads don't block each other, they're preemptive. Users don't schedule them, they run automatically when the preemption signal is received or when the current task is blocking
users don't have to do any scheduling themselves
sri well, a blocking syscall would block all tasks in the same thread 18:49
whiteknight yes, that's true. But then you wouldn't make blocking syscalls 18:50
sri right, that's what i mean, you have to be aware of it and use non-blocking syscalls instead
whiteknight and internally, things like the IO system will automatically be able to convert blocking operations into nonblocking+context switch
sri aaah 18:51
that's very important information :)
whiteknight Like if you do a socket.recv, it wouldn't block if you had data in the buffer, but would block if you had to recv. So it would be smart enough to do a context switch while the syscall was waiting
theoretically, in the future, hand-waving
sri well... that changes everything of course
whiteknight see? We're not completely incompetent 18:52
we just appear that way on the internet
sri pats whiteknight on the back
whiteknight so that's how you to 10k sockets with parrot: Create tasks, use non-blocking IO requests, and let the scheduler keep things moving while they wait 18:54
async io requests create callbacks, which become new tasks to pick up where the request left off
sri still leaves open the question of syscalls without non-blocking analog... fs I/O 18:55
whiteknight most of our platforms have non-blocking filesystem io, just not in a standard way with a standard api
so putting that in place is going to be harder
sri (libuv has a separate thread pool for that)
whiteknight that's an idea we've kicked around as well
of course if you serialize multiple blocking requests to a single requests thread, you still end up with a bottleneck 18:56
sri right, it seems to work for node.js though
sri wonders how erlang handles the problem 18:57
whiteknight sure, there are ways we can make it work if that's the direction we choose 18:58
erlang probably does something close to what I said: use non-blocking requests as much as possible 18:59
sri quite ambitious plans, but i'm intrigued 19:05
19:14 autark joined 19:25 wagle joined 20:03 wagle joined
dalek p/toqast: c07af7a | jnthn++ | src/ops/nqp.ops:
RSA should also report true in nqp::islist, otherwise the result of split does not count as a list, which caused some breakage.
 20:05
kudo/nqpqast: 9c1534d | jnthn++ | tools/build/NQP_REVISION:
Update NQP_REVISION to ease testing.
 20:10
20:27 cotto joined 20:34 lucian joined
dalek p/toqast: c0fbe4d | jnthn++ | src/6model/reprs/KnowHOWREPR.c:
Add missing mark in KnowHOWREPR that could lead to segfaults and other corruption.
 20:39
kudo/nqpqast: b79e158 | jnthn++ | tools/build/NQP_REVISION:
Another revision bump.
 21:07
kudo/nqpqast: 119716f | jnthn++ | src/core/Exception.pm:
Fix is_runtime to unbust backtrace stuff in some cases.
 21:12
23:04 lucian joined 23:29 benabik joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!