Parrot 3.9.0 "Archaeopteryx" | parrot.org | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 19 October 2011.
dalek kudo/nom: ef4702d | Coke++ | t/spectest.data:
track failure modes
 00:17
00:31 rfw joined 01:59 whiteknight joined
whiteknight good evening again, #parrot 02:00
dukeleto ~~ 02:24
Tene Definitely evening again. 04:02
04:57 nopaste joined 04:58 TonyC joined 05:32 dafrito joined 06:07 alin joined 07:39 zby_home joined 08:28 mj41 joined 08:54 cosimo joined 08:55 alin joined 09:19 lucian joined 11:52 bluescreen joined 12:11 Psyche^ joined
dalek kudo/bigint: 29eeaf8 | moritz++ | src/Perl6/ (2 files):
start to generate bigints in the actions.

Does not compile the setting yet		 12:47
kudo/bigint: e2c2af5 | moritz++ | src/Perl6/Actions.pm:
switch radcalc to bigints

now dies in quote_escape:sym<hex>
p/bigint: 05d6ab2 | moritz++ | / (3 files):
add nqp::isbig_I op, and nqp:: variants for bigint <-> str conversion
 13:10
p/bigint: 9647ef9 | moritz++ | src/HLL/Actions.pm:
make ints_to_string a method, so that it can be overridden in subclasses
13:17 mj41 joined
dalek kudo/bigint: e68268b | moritz++ | src/Perl6/Actions.pm:
fix hex escapes; restore compilation of the setting
 13:25
rrot/documentation_revisions: 2542f27 | (Alvis Yardley)++ | docs/README_revisions.pod:
Added Copyright clause to the top of README_revisions.pod
 13:36
rrot/documentation_revisions: f45e760 | (Alvis Yardley)++ | MANIFEST:
update manifest
rrot/threads: 51f564d | nine++ | / (3 files):
Fix eternal sleep of threads if main thread slept as well

When the main thread goes to sleep as well, on wakeup it only checked it's own alarms. Threads didn't get a chance to check theirs. Moved thread notification code to Parrot_thread_notify_threads and calling it now in both appropriate places.		 13:37
13:42 schmooster joined
dalek kudo/bigint: b147776 | moritz++ | src/Perl6/SymbolTable.pm:
move bigint serialization to the proper place
 13:54
13:54 smash joined
smash hello everyone 13:55
nine hello smash
15:01 whiteknight joined 15:04 Tene joined
whiteknight good morning, #parrot 15:07
mls hi whiteknight! 15:18
tadzik hello whiteknight
mls that sub autoclosing is dangerous ground: see trac.parrot.org/parrot/ticket/1550 15:23
and rt.perl.org:443/rt3/Public/Bug/Dis...l?id=56398
nevertheless, I believe the current implementation is wrong, as storing a context in the sub doesn't work for multi-threaded apps 15:24
the same is true for the current strategy that rakudo implements
NotFound mls: I think several things that are stored in the sub should be in the context. 15:31
Because the problem is that there is no "the" sub. There is an initial Sub object, and there are others created from it by newclosure 15:32
mls yes, that's one of the problems the current implementation has. :outer() just refers to one sub, but there may be multiple clones of it 15:39
whiteknight The Sub is, or should be, only a PMC wrapper around immutable bytecode
unfortunately, the Sub PMC holds a lot of state information that it shouldn't hold 15:40
mls how to you create a capture then? Is that a new PMC that stores both the sub PMC and the ctx PMC?
or is a capture just the ctx? 15:41
(ah no, can't be) 15:42
whiteknight Yeah, I think a capture is a new PMC type that is a Sub/Ctx pair 15:43
NotFound Is a clone of the Sub plus some tweaking with the lexicals
whiteknight or a Sub/Ctx/Outer tuple
right now, a capture is a Sub. In the future I want it to be a new thing
NotFound src/sub.c -> Parrot_sub_new_closure
mls Wasn't there a closure pmc some time in the past? searching... 15:44
whiteknight I think closure is a subclass of Sub
NotFound PMC * const clos_pmc = VTABLE_clone(interp, sub_pmc); 15:45
That's not a subclass
I think there used to be one, yes. 15:46
A Capture, by the way, is a completely unrelated thing. 15:47
mls Capture? I think Captures are only used in PCT
NotFound I don't know who uses it, but is a native PMC. 15:48
mls A capture contains both an array and a hash. It's meant to hold arguments for function calls
whiteknight right, so that's not the same thing as a lexical closure 15:49
I closure should be a Sub/Ctx tuple. Right now, any Sub can be a closure by setting ->outer_ctx
and that's a terrible design
mls It's a bit wasteful
whiteknight a bit 15:50
mls TT # 188 - Remove deprecatd Closure PMC 15:51
commit 02d9e438e74642dac4002d7ea594e62d898f193b
NotFound The problem for me is the mixing of static information about the sub code and properties and dynamic runtime data. 15:53
Cloning the static info makes no sense. 15:54
whiteknight right, and a lot of that data in the Sub is compile-time metadata that is used by the packfile loader to construct classes and namespaces
mls yes, but that's probably just an optimization that saves memory. We also need to fix the problems of the current implementation 15:58
NotFound Making copies of static things saves memory?
mls looking at the code rakudo produces it seems to me that it tries to workaround some of parrot's issues 15:59
whiteknight mls: how urgent is this for Rakudo?
I wasn't planning to work on Sub stuff for a little while, but I can get started early if Rakudo needs it
NotFound Yeah, they like to workaround issues instead of fixing them, and later complain if the fixes for the issues break their workarounds. 16:00
mls how urgent... hard to tell, it currently gives wrong results in some cases. I don't know if that blocks some real life code
NotFound: that's exactly how I wanted to continue
it's a shame that the problems aren't solved at the parrot level 16:01
whiteknight mls: they should be. It's a matter of limited time and resources
so let's fix them at the Parrot level. We're going to create a new LexicalClosure PMC type, and start refactoring some of the Sub data into that type instead
NotFound I still don't know what the problem is. Using newclosure things works like I think are intended to. 16:02
whiteknight then the capture_lex opcode will return a LexicalClosure instead of a Sub clone
LexicalClosure.invoke will set up the lexical environment, stealing some code from Sub.invoke
NotFound The thing is ugly, but seems to work.
mls sounds nice, but I don't see how that solves the current autoclose problems 16:03
whiteknight I think we don't want to clone subs anymore, because Subs should start to become immutable. We can clone LexicalClosures as much as we want
NotFound mls: Is a real problem, or just that some people is able to write code with ugly results?
whiteknight mls: LexicalClosure can perform dynamic lookups for the :outer, not be tied to the Sub
mls it's real perl6 code that returns wrong results 16:04
NotFound "If I a clone a Sub in some way, doesn't do what I expected". Then maybe your expectations were wrong.
whiteknight LexicalClosure can look through the scopes for the first instance of the :outer sub, or the first instance of a LexicalClosure that points to the :outer sub
NotFound: My point is that Subs should be pointers to immutable bytecode, and cloning a Sub shouldn't happen 16:05
cloning an immutable is just a pointer copy
mls whiteknight: that's also true for normal subs
you can call a normal sub with an outer that's supposed to go to a lexical closure
NotFound whiteknight: I agree on that, but before redesigning that machinery we should know what are the expectations and if they are right. 16:06
And even better, have some test cases. 16:07
whiteknight bleh. Lexicals are such a pain in the ass
mls yeah, we must be sure that we don't break anything. What worries me is that pmichaud++ spoke against looking at the call chain when resolving the outer ctx 16:08
clould also be that the implementation at that time was simply wrong 16:09
NotFound What call chain? At the outer during creation of the closure, ot at the inner usage?
mls autoclose
everything else doesn't have problems ;)
NotFound Ah, now I see why I haven't see the problem. 16:10
mls so you call a sub that's not a closure. The sub has :outer() defined, so you have to find a nice outer_ctx to link to
NotFound What's autoclose for? When calling the inner without the outer being run?
mls inner without the outer being run is just an edge case 16:11
that's when the current code creates a "dummy" context
the normal case is that you call a sub, and nobody has done a capture_lex on the sub 16:12
NotFound I still don't know why that ability is desired.
mls (so pretty much every call)
ok, not every call. every call that has :outer set 16:14
it happens when you nest "sub" definitions 16:15
I don't know if autoclose is really needed, and reading rt.perl.org:443/rt3/Public/Bug/Dis...l?id=56398 also didn't really help me 16:17
NotFound I shouldn't be needed it you can newclosure in every appropiate usage. 16:19
mls wait, it's not also about nested subs, but also about nested blocks (i.e. {} ), as blocks are emulated with subs to get a new lexpad 16:20
NotFound For example the rule in winxed is simple: inner are always anonymous and newclosure is called when used (if it uses lexcials) 16:21
So the problem is that suchs subs should be able to be just called and always know what you want to do? Sounds hard. 16:23
mls yeah. Bob Rogers also argus that parrot can't always know what the programmer wants 16:24
argues
NotFound If the .sub implementing a block must do the right thing in recursive call, using a static outer doesn't look right, certainly. 16:25
But I don't think parrot machinery can know what to do without some hints from the language. 16:26
mls yeah. I wonder if useing the first match from the dynamic call chain is always correct
whiteknight it probably isn't 16:27
mls dunno. please show me a counter example ;)
whiteknight I think NotFound is correct here: let's get rid of auto-closing and always force use the newclosure opcode when we want closing
NotFound The dynamic call chain can be completely unrelated if you return a closure and pass it to some unrelated code.
mls but then we *have* a closure, so we have the correct call chain 16:28
NotFound Ok. Then, again, is the language who knows how that sub can and will be used. 16:29
whiteknight or we get rid of auto-close for the common case, and add a invoke_as_closure opcode to close the target with the given outer ctx 16:40
but that's going to require some of the PCC changes I've been talking about
nopaste "NotFound" at 192.168.1.3 pasted "inner block-alike in winxed" (19 lines) at nopaste.snit.ch/94405 16:41
NotFound This example does a newclosure in the first usage of the block during the current outer invocation, and then reuses it. 16:42
(the newclosure is done automatically by winxed) 16:43
Using only a P register.
Now that I think about it: if we have a Capture PMC intended to hold things such as arguments for function calls, why we don't use it for function calls? 16:54
Oh, right, it contains only PMC. 16:58
cotto ~~ 17:01
nine whiteknight: are you up to date on my latest commits on threads? 17:06
whiteknight nine: not even close
nine: I've seen your recent commits on github, but I haven't pulled them or played with them myself 17:09
nine whiteknight: in short, alarms are handled by the main thread which notifies all other threads by writing to pipes on which they may block (if they're asleep). They then use the same Parrot_cx_check_alarms as the main thread. 17:10
whiteknight that's actually a pretty good idea. I like it
nine whiteknight: of course it's POSIX only again. But I guess there is some drop in replacement for the pipe thing on Windows. 17:12
And the main thread writes to the pipe unconditionally so it may overflow. Have to find a thread safe way to make this depend on the thread actually waiting. 17:14
whiteknight how does the main thread keep track of all the pipes? Just a huge array of pipe handles? 17:20
nine It keeps track of interpreters which are stored in a PMC* array. That's the reason why for now it only creates up to 4 threads. I tried storing them in a ResizablePMCArray but gave up after a couple of hours fighting with the GC. In any case I think a simple double linked list would be the right structure to use anyway. 17:21
whiteknight yes, I was thinking that too 17:22
nine The pipe is stored in the interpreter's thread_data (only on POSIX)
whiteknight okay
nine Is there some linked list implementation in Parrot that I could use?
whiteknight check out src/list.c
wait, that might be old
yeah, src/list.c 17:24
nine Looks good :) 17:28
What I really hate is, that I don't have any tests yet. I have my moretasks.pir but it's hard to turn it into real tests. 17:31
whiteknight yeah, it's hard to write tests for basic things like this, without relying on some kind of ordering guarantee, or timeout guarantee, etc 17:34
dalek nxed: ef52f3a | NotFound++ | / (2 files):
new make target 'testdebug':

also in debug mode, in order to exercise all assertions. Improve t/harness to handle the needed options
nine whiteknight: or things like looking in top if the parrot process really uses more than 100 % CPU ;)
NotFound Talking abou testing basic things... ;)
whiteknight nine: And even that requires that the OS kernel has scheduled you on two different cores, which it isn't guaranteed to do 17:38
nine Lot's of things I can't do or rely on. So I will just start with the things I _can_ do. Like kicking myself to just start it :) 17:40
whiteknight I'm probably going to be getting a new computer soon. When I do, I'll keep windows on one partition and install linux on the other. That way I'll have a dev/testing environment to do windows crap in 17:45
17:55 tty234 joined
nine Interestingly sharing globals between threads works well enough that I have not gotten any strange behaviour or errors yet. 18:15
whiteknight wow 18:20
18:21 jsut joined
nine Which also means that I do not even have a somewhat reliable manual test case for seeing if the messaging stuff will actually improve things. 18:22
whiteknight heh
dalek kudo/bigint: 3d97eda | moritz++ | src/core/Int.pm:
use new nqp::pow_I opcode
 18:26
kudo/bigint: 7c6a4a6 | moritz++ | src/Perl6/Actions.pm:
use add_numeric_constant for numeric constant
kudo/bigint: 933a1a5 | moritz++ | src/Perl6/Actions.pm:
document some assumptions; remove unused variable

no functional changes
moritz niecza: say 99 ** 10
p6eval niecza v11-22-gbebf7a0: OUTPUTĀ«90438207500880449001ā¤Ā»
whiteknight nine 18:29
nine: Just do your best to make things work, we will have a lot of time to find and fix bugs, write tests, etc
dalek rrot/threads: afb9539 | nine++ | / (2 files):
Add some minimal testing for threading functionality
 18:30
18:33 bluescreen_ joined 18:37 bluescreen joined
dukeleto ~~ 18:39
whiteknight hello duke 18:42
nine whiteknight: you said that library and class cloning has always been a source of problems?
whiteknight nine: the interp globals have always been the problem, especially deep-cloning of namespaces and the class hash 18:43
nine I get a "src/pmc/parrotinterpreter.c:128: failed assertion 'VTABLE_exists_keyed_str(d, d->class_hash, class_name)'" just by including test_more.pir in my test file... 18:44
So, do we need to clone these globals?
whiteknight nine: The problem is that classes are stored in a hash in the interp. When you create a new interp, you need to either clone those classes or you need to reference them in the other interp 18:47
if you clone them, you lose the link and updates to one aren't reflected in the other, etc 18:48
nine without cloning I'll get thread safety issues. But my gut says that this is probably not performance sensitive, so some locking might be ok. 18:49
whiteknight the long-term goal is to use those read-only proxies. You want to be able to update class metaobject definitions, just not from every thread that could ever touch it 18:50
18:55 contingencyplan joined
whiteknight plus, that should be much much cheaper than deep-cloning the entire set of all classes 19:01
nine I guess this is read mostly stuff anyway
whiteknight if we restrict class lookup to go through a few opcodes, those opcodes can lazily create proxies on the fly as necessary 19:02
although that might require a cleanup of the class store and lookup mechanisms 19:03
nine why not create the proxies on thread creation?
whiteknight we could, but what if we have a HLL like Rakudo with hundreds of classes, but a thread only needs to play with a few of them? Lazy creation is cheaper overall but we don't need to do it immediately 19:04
if eager proxy creation is easier, do that first
nine So I'll leave the library problem for the time being and go directly to proxies and messaging. 19:05
whiteknight wow, that would be awesome. I don't suspect libraries are going to be too difficult 19:06
although maybe I don't understand the problem with them
but yes, I think messaging is going to be a key component. Proxies are just a convenience that allows us to avoid locking 19:07
nine Well cloning seems to not work at all. Since I'll replace it by proxies in the long run anyway, I'll come back to this problem as soon as I have the tools to solve it ;)
whiteknight nine: You're in school, right? Might you be interested in GSoC next year?
nine whiteknight: in school yes. But I do already have a (even nicely compensated) full time job. So I'm not really in the target group. 19:08
Also, I'm not sure if I'll do a master's degree afterwards. So it may be that in July my studiy will be done. 19:10
whiteknight nine: Okay. I just want to make sure hard-working students have opportunity to get money
nine So, now's the time to think really detailed about how to implement messaging. 19:13
But for today, I'll just go home. Good night #parrot 19:27
whiteknight goodnight, nine
19:39 bluescreen joined
dalek kudo/bigint: 3014678 | moritz++ | src/Perl6/Actions.pm:
fix calling of radcalc for radix literals; make them produce the right type, I hope
 20:04
nine whiteknight: got to do some thinking on my way home on the bicycle 20:22
whiteknight: what do you think about adding a ResizablePMCArray to the Task PMC. The main thread pushes the PMCs it wants to share with the Task. When the task gets scheduled, the messaging and proxy stuff gets set up for these PMCs. Has the nice side effect of those PMCs still being referenced from somewhere within the main thread, so it's GC knows they are used. 20:24
whiteknight nine: Replace "ResizablePMCArray" with "concurrency-friendly mailbox" and you have a deal 20:25
nine: We want to be able to share more data than just the things we know about when the task is created. Any message received through a mailbox should be properly proxies
proxied
nine whiteknight: If the Task knows that it's actually run as a thread on it's own interp, when it gets scheduled instead of executing the attached sub, it reads the mailbox and updates PMCs. This way I can reuse the whole green_thread infrastructure for the mailbox stuff 20:26
whiteknight and then the mailbox basically becomes a one-stop shop "ProxyFactory" type, and nothing else needs to care about it
yeah, it's a good idea
the more you can reuse and integrate, the better
nine I see the mailbox and the structure referencing the shared PMCs as two different things. The structure is just to have an interface for telling which PMCs to share and for making the GC happy. Sharing additional stuff after scheduling the task could get done the same way, by just VTABLE_pushing onto the Task. 20:29
whiteknight pushing onto the task or to task.mailbox, sure 20:31
pushing into the mailbox, the mailbox will see that the target thread is different from the originating thread and will mark the PMC as shared for GC. On the receiving end, the mailbox creates a read-only proxy to avoid concurrency issues 20:32
er, contention issues
the mailbox basically contains all the locking and concurrency knowledge of the system. Everything else can be simple, by relying on a complex mailbox mechanism 20:33
basically, this design trades a little bit of locking everywhere for no locking, but robust mailboxes 20:34
nine Probably the most interesting thing will be the format of an update message. Essentially it has to contain a reference to the target PMC, the VTABLE method to call and the parameters to this call.
whiteknight nine: Yes, that will be interesting. Of course, if the message is a Task, and the mailbox just schedules it, we can just pass a closure 20:35
nine whiteknight: that's what I was just thinking. We just have to move execution of the updating OP to the data owning thread. Really just a very short task :) 20:36
whiteknight nine: I like the way you think!
nine So....the mailbox actually could be the other thread's task queue. 20:37
whiteknight so long as bytecode is immutable, that process becomes relatively easy
nine: possibly, but it does serve a slightly different purpose. The mailbox can take messages that aren't tasks
nine such as?
whiteknight nine: Just any random object that I want to share
I push it into a mailbox and you get a read-only proxy reference to it 20:38
Basically, we have two types of "messages": The data message and the task message 20:39
unless you think that's overkill
nine That's why I think the mailbox is quite distinct from the list of shared PMCs
whiteknight okay, I think I see what you are saying 20:40
you want to keep a list of shared PMCs as something like a reference count for GC
nine The list of shared PMCs could really just be a ResizablePMCArray on the task, since it's only written by the thread owning the Task PMC. The mailbox for updates could really just be the task queue, which I would just have to make thread safe. 20:41
whiteknight yeah, okay. I get that now. I misunderstood it earlier
nine whiteknight: yes, it's two things in one: the interface for sharing PMCs and the structure holding the references for the GC
whiteknight yes, a list of shared PMCs is a good idea
the alternative is to add another INTVAL field to the PMC header as a reference count
but that's ugly, and eats a lot of memory for the common single-threaded case 20:42
nine exactly
the ResizablePMCArray is dirt cheap code wise, uses proven code and does not touch the single-threaded case
whiteknight I like what you have to say, and would like to subscribe to your newsletter 20:44
nine The only part of this which I don't know how to implement is how to create the closure that does the actual updating 20:52
20:54 ambs joined
whiteknight if it's code we need, we can create a function in the frontend and store a reference to it in the scheduler at startup 20:54
all we would need to do is create a CallContext with the values we have, and pass that to the stored routine reference 20:55
or, we can call C code from PIR using an NCI wrapper 20:58
whichever you are more comfortable writing in
nine But an update could be anything like pushing a PMC onto a shared PMCArray. Essentially I need the code in the child thread trying to update with just the Proxy replaced by the real PMC 21:04
whiteknight right, you pass in a Sub that takes arguments and operates on those arguments. Then the message you pass is that Sub plus all the necessary args 21:08
the mailbox can look at each proxy, unwrapping each if it's on home turf
nine Aah, just like the sharing of data, the HLL compiler can generate the updating subs. It does not have to be transparent. 21:10
Ok, I think in case I lose my network connection for a couple of weeks, I still could come up with a working system :) 21:16
whiteknight nine: where abouts do you live?
nine whiteknight: Linz/Austria
whiteknight oh, okay. pretty far away then 21:17
do you expect to lose your connection for a couple weeks?
nine No :) Just wanted to express, that I believe that I now know all the pieces of the puzzle and will be able to put them together 21:18
whiteknight awesome, that's great to hear. If you come to any parts that seem stupid while you're working, you don't have to do what I've said 21:19
nine Ok, so no really good night :) 21:27
whiteknight okay, I'm heading home myself now. I'll talk to you later
goodnight :)
21:41 donaldh joined 22:11 bluescreen joined 22:29 rfw joined 22:35 donaldh joined
dalek nxed: 75d4037 | NotFound++ | winxedst1.winxed:
Ooops. These were ===, not ==
 22:53
23:14 alvis joined 23:20 awwaiid joined 23:34 zby_home_ joined
dukeleto ~~ 23:36
tadzik ~~
23:50 whiteknight joined 23:56 kid51 joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!