Parrot 3.6.0 "P�jaros del Caribe" released | parrot.org | Log: irclog.perlgeek.de/parrot/today | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 13 August 2011.
dalek rrot: 9114095 | Coke++ | / (9 files):
prep for release 3.7.0
00:02
00:08 whiteknight joined
whiteknight good evening, #parrot 00:11
plobsing hi whiteknight! 00:21
whiteknight plobsing!
plobsing I like what you're doing with frontend/parrot2 00:22
seems very lorito-ish
Coke rant: t/pmc/threads.t is failing for me sporadically as I test the release.
Coke will delete the offending test before the next release. 00:23
plobsing whiteknight: I've been considering how to eliminate the top '__PARROT_ENTRY_MAIN__' entry in the stacktraces. rakudo deals with a very similar problem by rolling their own backtrace pretty-printer. 00:25
whiteknight Yeah, but it's going to appear there for all backtraces, not just the ones that are unhandled up to the frontend
I really think I want to find a way to pop the top context off the call graph 00:26
of course, we need to be able to return to it after the program exits, so it's tricky
plobsing do exception handlers survive tailcalls?
whiteknight I think so. But where would we return to? We do have some cleanup to do, though not much 00:27
maybe not 00:28
Maybe we can add a way to flag a CallContext as invisible for the purposes of stack traces
backtraces* 00:29
plobsing what cleanup do we need to do that global teardown can't do? 00:30
whiteknight maybe none. I think I only pop off the top handler 00:31
so that's trivial 00:32
We can tailcall.
plobsing I dislike the option of hidding the top frame magically. we *want* to show it when bad things happen in startup (eg: allocation problems) 00:33
whiteknight yeah, true
I've got a Rosella commit I'm brewing up, I'll test that tailcall idea in a minute, unless you want to play with it 00:34
ttbot Parrot 9114095c MSWin32-x86-multi-thread make error tt.taptinder.org/cmdinfo/44716
plobsing I'm playing with it now. Unfortunately, the stack appears to be surviving tailcalls. Which makes my head assplode. 00:35
whiteknight thats...weird 00:36
That suggests maybe the "tailcall optimization" isn't optimizing anything 00:37
plobsing oh, it's doing something, though I'm not sure what
whiteknight bleh 00:38
plobsing __PARROT_ENTRY_MAIN__ tailcalls main, which calls a, which tailcalls b
guess which frame gets mangled in the backtrace
you get 4 guesses and you may need all of them
whiteknight ..?
plobsing main 00:39
one of the funtions that *isn't* tailcalling
whiteknight FFFFUUUUUUU
nopaste it? 00:40
nopaste "plobsing" at 192.168.1.3 pasted "tailcall oddness" (21 lines) at nopaste.snit.ch/71437 00:41
"plobsing" at 192.168.1.3 pasted "tailcall oddness (results)" (6 lines) at nopaste.snit.ch/71438
dalek sella: 1ad0253 | Whiteknight++ | / (5 files):
flesh out a lot more details with Assert
whiteknight hate 00:42
00:42 rdesfo joined
plobsing so it looks like the tailcall option has a few yaks to be shaved to work. not as expedient as I'd hoped. 00:43
whiteknight I worry that fixing this is going to break tests
I love that kind of fixing 00:44
plobsing the kind of fixing that breaks tests?
lucian whiteknight: i love having that worry
Coke tailcall was not enabled everywhere at one point because it failed horribly in some cases.
lucian it means i have tests! :)
Coke given that .return actually was doing 3 or 4 different kinds of returning.
release is cut, grab it from ftp or github tag and give it a quick whirl. 00:45
whiteknight Coke++
Coke constructs a release announcement.
whiteknight i used to have this tailcall stuff figured out, but its been a while 00:48
plobsing so from what I can tell, tailcalling clobbers the tailcaller's caller's frame in the backtrace.
whiteknight oh yeah, that makes *perfect* sense 00:49
plobsing a calls b tailcalls c gives a clobbered
I can see how it might have made sense to someone at some point. 00:50
lucian the newer parrot release appears very slightly faster on my laptop 00:51
plobsing If we wanted TCO but worried about frames not showing up in the backtrace ("Hey! a never calls c, wtf!!!"), we'd want to clobber the parent frame so that the call graph is maintained towards the tip.
lucian i get a clean, build test roundtrip in 45s instead of 50s, consistently
plobsing: backtraces and TCO are tricky 00:52
plobsing my opinion: you ask for TCO, you're asking to drop the frame
whiteknight yes
lucian if TCO is always explicit in parrot, i agree
whiteknight more likely, it wasnt designed
just....bloop
oh look! where did all this code come from? 00:53
Coke wonders how cotto got the list of contributors for the 3.5.0 release. 00:54
lucian oh! i just remembered i forgot to flame parrot about lack of module system and debugging, in that blog post 00:55
whiteknight we have plumage 00:56
lucian that's a package manager, not a module system
whiteknight oh, gotcha
lucian a module system is something that lets you import things sanely
whiteknight we're working towards it 00:57
lucian i don't even know how much time i wasted tracking line numbers in a concatenated winxed file, and then mapping that to the original winxed file
whiteknight: anything specific? 00:58
plobsing lucian: are you looking for a file lookup mechanism? a static importation mechansim? a dynamic importation mechanism? all? other?
lucian plobsing: one good import mechanism, ideally dynamic 00:59
plobsing namespaces have that, somewhat
lucian a little
i can't sanely import c in b, and then b in a 01:00
whiteknight PackfileView moves pretty far in that direction
and namespaces need to be refactored with fire
plobsing firey chainsaws
lucian i have all my files in one namespace
whiteknight The big problem, and I've been sitting on an angry blog post about this, is that all the information about namespaces is stored in Sub objects and recreated from there by the packfile loader
lucian at build time, they get concatenated in one "module", to produce a single .pbc 01:01
whiteknight the packfile loader is a big jerk, namespace is trying to be a solution to an absurd number of resulting problems, and everything else suffers
lucian and the line numbers on exceptions are of the concatenated file
whiteknight: what do namespaces have to do with subs? 01:02
whiteknight lucian: good question
plobsing lucian: subs think they own their spot in the namespace
in a has-a sense
whiteknight plobsing: worse than that, Subs think they build and control the namespaces
lucian plobsing: but... subs should be just referenced names ...
lucian gives up 01:03
whiteknight When IMCC compiles stuff, it stores information about a Sub in a single structure, then dumps all that info into a single Sub PMC and freezes it
plobsing names should just reference subs, which should be anonymous
lucian plobsing: of course, that's what happens everywhere :)
whiteknight actually, let me finish and publish that blog post, it answers all the questions
lucian plobsing: well, python and js anon fns are optionally named
whiteknight: ok 01:04
plobsing lucian: yes, subs have something they call themselves. but that doesn't enforce anything externally
lucian nods 01:05
in js, you can do var bla = function bla_foo()...
in python, def foo():pass does the same thing, essentially
plobsing: the main usage for names on fns that i've found is debugging 01:06
plobsing the problem with subs in parrot is that they are too convenient a place to hang information to not become overrun by cruft 01:07
lucian i see
whiteknight: what does "freeze" mean for a PMC? 01:09
whiteknight lucian: pickle
lucian ok
somehow i assumed "make immutable" 01:10
Coke we have commits from 2 different jim keenans email addresses. 01:11
lucian goes to sleep 01:13
whiteknight begun, the clone coding has 01:14
plobsing: do you know where those frames are getting squashed? Maybe it's an easy fix 01:16
plobsing whiteknight: I'm looking into it.
but it involves pcc and exceptions, 2 subsystems I am not 100% comfortable with. 01:17
on an unrelated note, I benchmarked parrot2 vs master on rakudo startup
not a statistically significant improvement 01:18
but I think it's worthwhile just for the cleanup it allows 01:19
whiteknight It wouldn't affect rakudo at all. I only redid the parrot frontend, not pbc_to_exe 01:20
It shouldn't anyway
plobsing I used the 'parrot perl6.pbc' form for testing
whiteknight oh, okay 01:21
I saw up to about 3% improvement on startup when there were large amounts of :init subs 01:22
but a small slowdown when there were few
plobsing rakudo has 631 :init subs and 33 :load subs
or vice-versa
I remember the numbers, but I got the order I tested jumbled 01:23
kid51_at_dinner Coke: That probably reflects whether I committed from Linux or Mac.
plobsing whiteknight: how were you testing?
whiteknight it's a small improvement, and only for startup, but you are right that there is some improved code
I had a program with 1000 copy+pasted empty :init subs, and a shell script that ran it 1000 times 01:24
and I varied the number of :init subs for each run
plobsing ah, that's a more pure test than mine
whiteknight 10, 100, 1000, etc
yeah
it's only startup performance, so I had to isolate
plobsing mine was 100 runs of "perl6 -e ''" 01:25
whiteknight I had some callgrind output. It's amazing how much other stuff weighs down the startup profile
plobsing I'd put money on the allocator and deserialization dominating
whiteknight I have to look at the profiles again 01:26
okay, I'm out for the night. feel free to push anything to that branch that you ant 01:31
want
I'll catch up in the morning 01:32
kid51 Coke: That different email address in the commit log represents the first time ever that I conducted a git pull and merge strictly thru the github GUI. 01:35
dalek website: coke++ | Parrot 3.7.0 "Wanda" released 01:38
website: www.parrot.org/news/2011/Parrot-3.7.0
Coke thinks the release logistics are done. 01:40
oh, channel. 01:41
moderator Parrot 3.7.0 "Wanda" | parrot.org | Log: irclog.perlgeek.de/parrot/today | #parrotsketch meeting Tuesday 19:30 UTC 01:41
kid51 Coke++ 01:44
Coke soh_cah_toa++ #tag, you're it 01:47
soh_cah_toa :) 01:48
Coke: about how much time did you spend on release manager related tasks? i mean, around what time should i start preparing? a week? two weeks? more? 01:51
Coke I was lucky and NEWS was kept up to date. 01:56
(or so I hope. ;)
I woudl go through the list in the next week. make sure you have access to the machines & websites listed. 01:57
give an hour or 2 for that.
it took me about ... 3 hours, on and off tonight to actually cut the release.
and most of that time was on a slow linux box. 01:58
(doing make test)
soh_cah_toa ok, that's do-able
Coke oh! be sure to run the make with -j if you can, and set TEST_JOBS if you haven't.
soh_cah_toa of course, always :)
Coke I was just cutting and pasting the directions, which did not. if you think for yourself, you'll go fine. ;)
soh_cah_toa :) 01:59
dafrito Oh noes! The OTTO/WANDA dialogue breaks the parrot.org/ layout! 02:06
dalek TT #2161 closed by jkeenan++: Whitespace confusion in PCT docs 02:10
TT #2161: trac.parrot.org/parrot/ticket/2161
kid51 dafrito: I don't see what's wrong. 02:12
Coke used pretty basic html. 02:17
ah, too wide? fixable. 02:18
better? 02:19
soh_cah_toa yup
02:31 woosley joined 02:33 jsut joined
dalek kudo/nom: 22d4296 | Coke++ | t/spectest.data:
run this (fudged) spectest
02:58
03:05 theory joined 03:19 cotto joined
cotto ~~ 03:21
dalek TT #2180 created by bubaflub++: examples/nci/ls.pir broken 03:29
TT #2180: trac.parrot.org/parrot/ticket/2180
03:34 AzureSto_ joined 03:35 nbrown joined, rdesfo joined
dalek rrot/whiteknight/frontend_parrot2: a78f000 | plobsing++ | frontend/parrot2/main.c:
pack struct more aggressively on lp64 machines
03:38
rrot/whiteknight/frontend_parrot2: 68ee9d8 | plobsing++ | frontend/parrot2/prt0.pir:
tailcall into main (should eliminate prt call frame). fixes 2 coretest failures.
rrot/whiteknight/frontend_parrot2: cbdfab6 | plobsing++ | compilers/imcc/imc (3 files):
flex/bison stupidity
rrot/whiteknight/frontend_parrot2: 28df851 | plobsing++ | frontend/parrot2/prt0.pir:
avoid printing trailing empty line
rrot/whiteknight/frontend_parrot2: 087598d | plobsing++ | frontend/parrot2/prt0.pir:
for whatever reason, handlers must be poped from the context in which they were pushed
plobsing msg whiteknight I've run into more funny business with tailcall. The problems I mentioned earlier only occur in --optimize builds (I suspect the pcc accessor macros/functions). 03:40
aloha OK. I'll deliver the message.
plobsing msg whiteknight I've pushed some commits which fix a number of coretest failures. Most of the remaining failures are accounted for by 2 pre-existing faults in parrot: (1) tailcall frames are only elided after the callee fetches its arguments (which it doesn't always do). (2) the way subs pretty print themselves for backtraces expects the throwing frame to still be the active one. 03:43
aloha OK. I'll deliver the message.
plobsing argh! after staring at my list of commits for 5 minutes before pushing, I only see the spelling error in my commit message when it hits the channel via dalek 03:48
04:06 daniel-s joined 05:06 Tene joined 05:15 wagle joined
dalek p: 6fe5a07 | moritz++ | tools/build/PARROT_REVISION:
bump PARROT_REVISION to 3.7.0
05:17
kudo/nom: c75ed52 | moritz++ | src/core/Str.pm:
respect sign in "-:16<AB>".Numeric
05:18
kudo/nom: f959eac | moritz++ | tools/build/NQP_REVISION:
require an NQP that requires parrot 3.7
cotto my slides are on mksig.org/slides/yapceu2011/ if anyone would like to review. 05:19
the talk is happening in about 7 hours 05:20
05:24 wagle joined
dukeleto ~~ for a minute 05:27
dukeleto looks at cotto++'s slides
dukeleto cries when he sees an .odp
cotto dukeleto, as opposed to ...?
dukeleto cotto: anything else :)
cotto: i am still going to look at it, because I like you :) 05:28
cotto powerpoint '98? ;)
thanks
moritz slides look fine 05:29
cotto moritz, thank you
aloha, clock?
aloha cotto: LAX: Tue, 22:29 PDT / CHI: Wed, 00:29 CDT / NYC: Wed, 01:29 EDT / UTC: Wed, 05:29 UTC / LON: Wed, 06:29 BST / BER: Wed, 07:29 CEST / TOK: Wed, 14:29 JST / SYD: Wed, 15:29 EST
dukeleto cotto: i like the visualizations describing the relation between ops2c and nqp. Nicely done 05:31
cotto: nice work and good luck. Find some new parrot hackers :) 05:32
cotto I'm excited about tomorrow. I'm going to be attending an all-day Perl 6 session lead by jnthn++
time to wander around town and head over to the venue 05:34
sorear How is Perl6 doing in the public (well, YAPC) conciousness these days? 05:35
When I first expressed interest in Perl6 a little over a year ago, I was laughed at.
best slide format (as a consumer): probably pdf 05:36
05:39 SHODAN joined 05:48 fperrad joined
tadzik sorear: depends on who you talk to 05:55
sorear: but sometimes they're actually "oh really? That's cool, I'd really like to get into it" 05:56
the closer to Perl 5 core they are the more reluctant they seem to be
06:04 cotto joined
cotto sorear, Perl 6 has had excellent representation at yapc::eu. Damian talked about it as the 2nd day's keynote and there are quite a few Perl 6 sessions. 06:06
06:28 squiggy joined
squiggy hello which model can parse the POD include =row =cell format? 06:29
Parrot doc of ch04_pge.pod have many such format character 06:30
I feel the POD in Parrot is different with Perl5 POD.. 06:31
docs.parrot.org/parrot/latest/html/...e.pod.html 06:33
also ignore this content
06:35 cottoo joined
sorear squiggy: Pod::PseudoPod::Latex 06:38
iirc
it's one of O'Reilly's submission formats, and it's also quite similar to POD6
tadzik might be able to parse it 06:39
as well as P:PP:L
squiggy ::O 06:42
:U
I have not Latex Viewer. I used Pod:;Simple I think I should look example of PGE not help 06:45
I only find two valid pg files in folder of Parrot.3.7.0 It is too short..I should download Rakudo sourcecode to find..:O 06:50
sorear um, NO
Pod::Simple is a Pod processor
IT CANNOT HANDLE PSEUDOPOD
Pod and PseudoPod are NOT COMPATIBLE
get Pod::PseudoPod 06:51
dalek rrot: 7a9e7ad | NotFound++ | examples/nci/ls.pir:
update example ls.pir to recent changes in NCI (no 't' type)
06:57
squiggy Pod::PsedoPod is based Pod::Simple... I have not lose hope with Pod::Simple :P:P 07:01
dalek rrot: 91518de | NotFound++ | examples/nci/ls.pir:
die nicely if opendir fails in example ls.pir
07:08
squiggy pmc write with PIR should run in same rapid with write in C 07:18
the PGE syntax is similar with Regexp::Grammars is it? 07:19
moritz similar, but still quite different 07:20
squiggy It is ok to parse Parrot POD with Pod::PseudoPod 07:21
ttbot Parrot 7a9e7ad4 MSWin32-x86-multi-thread make error tt.taptinder.org/cmdinfo/44771 07:23
squiggy Pod format of Parrot is different with Pod In Perl5? 07:41
ttbot Parrot 91518de1 MSWin32-x86-multi-thread make error tt.taptinder.org/cmdinfo/44788 07:43
sorear 23:50 <@sorear> Pod and PseudoPod are NOT COMPATIBLE 07:48
everything under book/ is written in PseudoPod, NOT POD
cotto Most of the POD in Parrot is plain POD, apart from book/ 07:49
07:49 mj41 joined 07:54 contingencyplan joined
squiggy o, thanks 07:54
cotto This is odd. Half of what Jesse Vincent is talking about for his 5.16 keynote sounds pretty close to what we're trying to do with Parrot. 07:55
moritz :-)
cotto it should be on yapc.tv at some point
I see a camera, so hopefully it's being recoded. 07:56
moritz maybe it's not so odd after all
cotto perhaps not. It's catching me off guard. 07:57
TimToady cotto: running on all VMs has been on my roadmap since the mid-90's actually, when I sat on the master's exam committee for someone who tried and failed to put Perl onto the JVM 08:00
dalek kudo/nom: 6cac7f1 | moritz++ | / (2 files):
implement p5chomp, p5chop (scalar case only)
08:13
08:31 cotto joined
cotto nine, ping 08:32
Is anyone around here interested in mentoring nine to help him get started working on threading? 08:34
sorear cotto: what does this entail 08:48
cotto sorear, probably just helping him get familiar with parrot guts and answering questions. 08:49
s/just//
familiarity with the current threading code could be helpful, but I'm not sure if there'll be anything worth saving. 08:50
btw, the Rakudo folks are sad that you couldn't make it. 08:51
sorear to yapc you mean? 08:55
yeah it went like this
me: hey, I got an offer to attend a Perl convention in Latvia 08:56
cotto yes
sorear mom: You've never been outside the city alone, this is way too much for you. No.
moritz sorear: so you need to attend YAPC::NA first 08:57
sorear: or something other inside the US
sorear: next YAPC::EU is in frankfurt, which means that the chances are very high that I can attend (it's only about 250km distance from $home) 08:58
cotto moritz, great
09:00 rfw joined 09:31 cotto joined 09:40 ligne joined
dalek kudo/nom: 43f9e11 | moritz++ | / (3 files):
fix split, run split-simple.t
09:43
kudo/nom: 6246bdf | moritz++ | / (2 files):
Cool.chr
09:58
10:50 Coke joined 10:54 cotto joined
cotto soh_cah_toa++ #holy mackerel that's a long gist 11:04
I may have to save that for the flight back home. I'm glad it's a long flight. 11:05
11:40 cotto joined 12:04 p6eval joined 12:07 whiteknight joined
atrodo =~ 12:08
whiteknight good morning, #parrot 12:09
msg soh_cah_toa I have taken a few looks at it. I've got some other work planned to do in IMCC as well, so maybe we can piggy-back on that while I'm "in the zone"
aloha OK. I'll deliver the message.
12:50 Kulag joined 12:52 contingencyplan joined, JimmyZ joined 12:57 bluescreen joined 13:06 rohit_nsit08 joined 13:08 rdesfo joined 13:17 ambs joined
cotto hio whiteknight 13:27
whiteknight good morning cotto 13:29
thanks for the email this morning. It's good to hear a concise list of what Rakudo wants and needs 13:30
and it's reassuring to think that they're asking for things that I'm actively pursuing
cotto whiteknight, awesome and a half 13:31
13:33 lucian joined
whiteknight I have become *extremely* annoyed recently by the state of the Sub PMC and related details 13:34
so I'm definitely going to work to improve those things
PerlJam whiteknight++
whiteknight the problem is that we are definitely going to run afoul of the deprecation policy here. Sub and it's minions are far to central to Parrot for major refactors to have no user-visible effects 13:35
we're going to end up with some pretty heavy branches hovering overhead, waiting to crash into master the day after 3.9 13:36
PerlJam Better that than dragging it out
whiteknight I feel like waiting until 3.9 *is* dragging it out 13:37
moritz whiteknight: got a link to that blog post? planet.parrotcode.org times out here
PerlJam Well, that no one did the analysis of Sub PMC that you did sooner has dragged it out too, but you can only do so much
whiteknight whiteknight.github.com/2011/08/15/sub_problems 13:38
moritz: And I have a follow up that should be posted within the hour
PerlJam whiteknight: the follow up outlines a way forward? or are there even *more* problems? :) 13:39
whiteknight outlines the first few steps I've been able to plan so far
there's a hell of a lot of work ahead of us, if we want to do this and do it right 13:40
considering we've already done it and done it wrong, I think we need to consider the alternative path
:)
PerlJam wonders how long before he starts seeing posts about parrot show up on thedailywtf 13:43
moritz PerlJam: parrot is big and has some messy cornerns, but overall it's not that bad 13:44
PerlJam moritz: If anonymize and genericize whiteknight's post on subs, it's quite like things I've seen on thedailywtf before. 13:46
s/If/If you/
atrodo whiteknight> Just to verify my understanding, mmd is just a pmc, right? No ops involved? 13:47
moritz PerlJam: dailywtf has usually far more blantly stupid code 13:48
whiteknight: one reason for having a namespace pointer in a sub PMC is for backtraces 13:52
whiteknight: though I suppose that problem could be solved differently too
whiteknight moritz: There's a pretty clear line between "old" parrot and more "modern" Parrot. The more modern parts of it are actually pretty nice and getting nicer. The older crufty parts are where the biggest problems come from 13:53
and unfortunately, these old crufty parts are the parts that that central to core operations, and form major segments of the internal structure
Code in src/sub.c, src/oo.c, src/namespace.c and some of it in src/debug.c is both horrible and absolutely fundamental 13:54
if you feel like dramatically lowering your opinion of Parrot, read those files 13:55
PerlJam I'd rather let you read them and blog the cliffnotes version for the rest of us :) 13:57
whiteknight PerlJam: Okay, here are the cliffnotes: HORRIBLENESS. Gnashing of teeth and weeping of children. Exeunt, stage right 13:58
moritz whiteknight: btw rakudo abuses the multi_signature pointer for storing a pointer to the Perl 6 Code object that's a wrapper for the parrot sub
atrodo weeping of children. that's a nice touch 13:59
moritz whiteknight: might be worth keeping in mind that HLLs might want such a pointer
13:59 rohit_nsit08 joined
PerlJam whiteknight: but there's no wailing or moaning? 13:59
moritz whiteknight: I've finally finished reading your excellent post. And yes, the factoring you describe in the end is certainly the way to go 14:03
whiteknight moritz: yeah, I talk to jnthn about that the other day. I think what we want to do is slim Sub down to bare-bones, and let HLLs provide custom wrappers. It isn't really possible now, but it will be
plobsing whiteknight: you missed a spot of ugliness on Sub PMC. someone noticed there were too many attributes but wanted to shove more crap into it anyways, so they used up all the PMC private flags. 14:04
whiteknight Having a pointer around with a stupid use, and forcing an HLL to abuse it is hardly a good long-term solution
moritz whiteknight: and I arrived at basically the same conclusions (decouple Sub from namespace, methods, vtables, multi, don't auto-create multis)
whiteknight plobsing: yes, the flags are another area of concern
plobsing: I want to change IMCC to build NameSpace and MultiSub (and maybe some Class) PMCs at compile time and serialize them in 14:05
that should cut out all this runtime startup garbage
plobsing that'll take some syntax changes
whiteknight you think?
When we see a .namespace directive we create a NameSpace PMC. Every sub thereafter gets jammed into it 14:06
rohit_nsit08 whiteknight: hello, sorry to interrupt but needed some quick help in using ".include " i'm getting the error that file I'm trying to include doesn't exist.
plobsing and how do you manage the case where multiple files want to "co-operate" to build the same namespace that eventually becomes a class?
whiteknight plobsing: at runtime, if we try to thaw a namespace that has the same name as one that already exists, we merge
VTABLE_set_pmc should be very useful there
same thing with MultiSub 14:07
dalek kudo/nom: 63b54d1 | moritz++ | NOMMAP.markdown:
add memory leak to NOMMAP punchlist
plobsing whiteknight: that sounds like magical namespace behaviour all over again
whiteknight When we thaw a namespace, we need to merge it into the global namespace tree anyway
we can't change all semantics, and we can't be breaking get_hll_global and related ops. We do still need a global namespace tree
JimmyZ wonders why rakudo doesn't use namespace 14:09
plobsing JimmyZ: read our namespace code and become enlightened. or enraged. or both.
whiteknight rohit_nsit08: what file are you trying to include?
moritz because it needs its own stuff, "as always"[tm]
whiteknight rage is the path to enlightenment 14:10
atrodo or destruction
whiteknight plobsing: Namespaces have string names that are supposed to be unique. I don't think merging namespaces with the same name is magical. They aren't things to themselves, they're just storage of other things by name
rohit_nsit08 I have a "jsArray.pir" which is extending parrots "hash" class
whiteknight rohit_nsit08: Where is jsArray.pir located? 14:11
it needs to be on the include path
rohit_nsit08 in the same directory
parrot's include path
?
plobsing whiteknight: if we want unique namespaces, maybe we should populate them at runtime
whiteknight that's basically what it is doing. If we don't serialize the NameSpace itself, we can serialize a ProtoNameSpace, and have easy one-step loading into the NameSpace 14:13
I'm fine with that too
what matters is the grouping of Subs under a single unique name. So long as we do the grouping at compile time, I don't care what form it takes
actually, ProtoNameSpace is probably a better idea, because it can contain a fully-qualified NameSpace name, a list of subs, and some other details. Create the NameSpace at runtime when HLL mappings are set up 14:14
14:15 dafrito joined
dalek kudo/nom: e0f9eee | Coke++ | t/spectest.data:
track (non) failure mode.
14:15
plobsing my concern with auto-merge is that it allows the PBC builder to think they have a reference to the object that is going to be the namespace (and thereby try to avoid by-name lookups of such). if there's a collision, that illusion is smashed to pieces. 14:17
whiteknight okay, so that just lends to the idea that we create something that isn't a NameSpace at compile time, but still create some kind of grouping there for Subs 14:19
plobsing yes
but be very explicit that it is *not* the namespace object
whiteknight We still want Subs in the same namespace during the same compilation to be stored in some kind of container, and we don't want Subs to contain their own storage keys
okay, I'm onboard with that
moritz I'm trying to be the heretic, and ask "do we need namespaces at all?" 14:20
whiteknight plobsing++ on the whiteknight/frontend_parrot2 fixes last night, by the way
moritz how many dynamic languages actually have namespaces?
whiteknight moritz: In the long run, I would say no
nopaste "coke" at 192.168.1.3 pasted "for rohit - include example:" (15 lines) at nopaste.snit.ch/71537
whiteknight moritz: but we have a *lot* of infrastructure that relies on it. Several dozen opcodes, several non-trivial semantics, etc 14:21
plobsing heh, I took the easy fixes and msg'd you about the hard-to-nigh-impossible fixes
atrodo I'm under the current opinion that we need containers, but namespaces are not it
whiteknight plobsing: yeah, those two faults are doozies
atrodo; we can always rename them to something different
plobsing the one might need changes to PCC. again. 14:22
whiteknight what NameSpaces are, in parrot, is a global tree of string-keyed containers
it's helpful to say "give me X.Y.Z"
atrodo whiteknight> But can you have containers without having them in the global tree?
whiteknight atrodo: Sure, but they lose a lot of usefulness
Coke rohit_nsit08: there's a very simple include example. 14:23
whiteknight atrodo: If I load in a third party library and need to initialize it, how do I do that? I need some kind of standard way to search for symbols
plobsing namespaces are handy for keeping our HLLs in their own little sandboxes unless they ask nicely to be let out
whiteknight Rosella, for instance, is going to require you to call the "Rosella.initialize_rosella" function to set up and initialize the library. We need to be able to quickly and uniquely find that symbol without having anything more than a .pbc
atrodo whiteknight> I'm just thinking some languages might not want to expose their entire tree to the world, and either hide parts or be marshaller of the symbols 14:24
whiteknight atrodo: yes, that's a good point too. Making the namespace tree much more flexible and dynamic is key
atrodo whiteknight> Yes
whiteknight the current implementation is not that. A future implemention should be
rohit_nsit08 Coke : I am basically trying to do "TclArray" thing to "jsArray" but It says that Class "jsArray" not found 14:25
whiteknight That would make, for example, exporting easier. I should be able to load namespaces in a packfile as subnodes of a given node, be able to load language-agnostic libraries directly into a particular HLL namespace,e tc
The only way to do that is to make NameSpaces in charge of themselves, not be puppets of the Subs they contain 14:26
because if you have 100 Subs and all of them point to namespace Foo, to move Foo around in the tree, or move those Subs around in the tree requires you to update 101 PMCs
Coke rohit_nsit08: ok. if the include is working, then you're not running the code, just making it available.
atrodo whiteknight> It's definitely upside down 14:27
whiteknight atrodo: no doubt
Coke presumably you'll need the appropriate ":init" or ":load" marker on a sub that is actually creating the class.
(I have meetings coming up for an hour, so be sure to help rohit out, folks.)
rohit_nsit08 Coke : hmm I'm having :load and :anon markers on it 14:28
what is the purpose of :init? 14:29
:init makes the subroutine to run before the main
Coke : worked 14:30
init was required
whiteknight :init runs when the file is a program. :load works when the file is a library 14:34
rohit_nsit08 whiteknight: hmm, assuming I have a loaded library . how will I use it than ? for example I have classes declared in a file and I want to declare objects of that class in my main program 14:41
whiteknight $P0 = load_bytecode "my_file.pbc"
no, wait
load_bytecode "my_file.pbc"
that will load in the file and execute all the :load functions
rohit_nsit08 got it, but load_bytecode can also be used without including the file first . am I correct 14:43
the only thing to keep in mind is to use :load marker with all the subroutines
whiteknight right
.include is whole-text inclusion
it's like copy+paste
and it's done at compile time
load_bytecode happens at runtime 14:44
rohit_nsit08 hmm I should use load_bytecode than
15:23 JimmyZ_ joined, theory joined 15:26 ambs_ joined 15:52 ReachingFarr joined 15:56 ambs_ joined
plobsing whiteknight: where is your proposal for splitting up get_params. I'm cooking up a solution to the problem of tailcalls being subverted by the callee, and think I can work that in at the same time. 16:03
whiteknight the get_params thing is going to be a pretty big refactor, new opcodes and all that crap
plobsing yes, but where is the proposal? 16:04
whiteknight whiteknight.github.com/2011/05/11/p...ments.html
and some practical details: whiteknight.github.com/2011/05/12/p...mings.html
plobsing thanks
whiteknight I would be thrilled if we started moving in this direction, even baby steps 16:06
ReachingFarr Is there a good high-level overview of the Parrot VM internals somewhere?
whiteknight ReachingFarr: I'm sure we can dig one up. What's your background? 16:07
plobsing ReachingFarr: sure, but where to start depends on your objective
whiteknight ReachingFarr: docs.parrot.org/parrot/latest/html/....pod.html. The whole docs.parrot.org website is *mostly* good 16:08
ReachingFarr whiteknight: Ph.D. student working on parallel programming languages.
16:08 rdesfo joined
whiteknight oh, nice! We need more of those things: Ph.D. students and working parallelism 16:08
ReachingFarr plobsing: I'm looking for a starting point for a new language + runtime system.
whiteknight ReachingFarr: As a disclaimer, before you get too deep into Parrot, we don't currently have workable threading 16:09
ReachingFarr whiteknight: Thanks for the link.
whiteknight We want it and are working on it, but we don't have it now
plobsing parrot's support for parallelism is... underwhelming
ReachingFarr whitenight: Ohh, ok. That is a good thing to know. I'd only been able to find a few blog posts and a wikibook entry on Parrot threading.
lucian plobsing: you can spawn several parrots :) 16:10
whiteknight And all that stuff is out of date
lucian ReachingFarr: the JVM is awesome with threads
whiteknight Parrot *will* be awesome too, when we get our act together
lucian ReachingFarr: but threads do suck, so that's not as useful as it sounds
ReachingFarr lucian: Ya, the other options I'm looking into are the Jikes RVM and using LLVM.
plobsing lucian: you can spawn multiple python processes too, but there are still many that will complain about the GIL.
lucian plobsing: meh
16:11 logie joined
lucian loves fork 16:11
plobsing loves perl -e 'fork while fork' (on somebody else's machine of course)
whiteknight ouch 16:12
ReachingFarr I know that no one can give a REAL answer to this question, but about how far away is Parrot from having support for native threads in terms of time and/or work?
whiteknight ReachingFarr: We have a bad system now, which is messy and not maintained/tested/loved. We are working to design a new system 16:13
so, we're essentially at the design stage, with some potential for code reuse in some places
lucian ReachingFarr: realistically, quite far. correct threading is hard (threads suck) 16:14
plobsing lucian: computers suck (FTFY)
lucian plobsing: ok, shared state sucks 16:15
ReachingFarr Well, it sounds like Parrot is probably not the right choice for this project right now.
Thanks for your help with my questions, and good luck with Parrot. I like what I've seen so far :-)
whiteknight ReachingFarr: if you find yourself with an urge to help implement a low-level VM threading system, you know where to find us :) 16:16
lucian ReachingFarr: if you don't mind, what are you looking to build?
whiteknight we need all the hands and minds we can get to make a great system
ReachingFarr whiteknight: Wish I could, but I have a grant proposal, classes, and this project to work on at the moment :-( 16:17
whiteknight I understand. Good luck!
ReachingFarr lucian: It's a runtime system for a new programming language. The runtime is structured very similarly to a multikernel. 16:18
lucian ReachingFarr: what's the language like?
whiteknight We've turned more potential new users away for lack of concurrency support than for any other single reason, I think 16:19
it's becoming a pretty big pain point
lucian whiteknight: really?
ReachingFarr lucian: Well, there are no threads.
lucian ReachingFarr: good :)
whiteknight: it doesn't seem to me like something people actually lack
ReachingFarr: what parallelism do you have instead? 16:20
ReachingFarr lucian: It is an OO + Actor model language. Each object is theoretically executing concurrently, and they can only communicate via message passing. The runtime system moves objects between cores, and a worker thread executes them. When messages are passed to a core-local object a direct call is made instead of expensive message passing. The idea is to avoid triggering the cache-coherency protocol, so each core uses a private chunk of memory. 16:21
lucian ReachingFarr: sounds like stackless & erlang 16:22
whiteknight that's almost exactly the kind of concurrency system I wanted to implement for Parrot
lucian ReachingFarr: lots of prior art out there on that
whiteknight well, I didn't go into details about cache coherency, but the actors dispatched to unerlying worker threads is exactly the system Parrot wants 16:23
ReachingFarr lucian: Kind of. Some of the initial ideas came from Erlang. Some of the additions are syntactic sugar. Like futures. But the futures also add a lot of areas for optimization. The main difference is the parallelism granularity. In Erlang you still have processes, and that is fine-grained as you can get. In this language you could theoretically execute every instruction in parallel (barring data dependencies).
lucian: That's why I've spent the summer reading :-) 16:24
lucian ReachingFarr: interesting, anyway
if i were you, i think i'd do it on the JVM
whiteknight ReachingFarr: where do you go to school?
lucian it's by far the most mature for that sort of thing
ReachingFarr lucian: I'm more interested in the LLVM, but the JVM is a contender. 16:25
whiteknight: University of Colorado
lucian ReachingFarr: with LLVM, you'll have to implement everything yourself
on the JVM, you'd be free to work on your language
on the other hand, you might have a little less control on its execution
plobsing lucian: there are JVMs that can have their hoods openned up. Jikes is popular for such modifications, IIRC. 16:26
lucian plobsing: indeed
ReachingFarr lucian: Yah, but the structure of the runtime is going to be fairly non-standard. I also need fast message passing and quick co-routines.
plobsing I recall reading that someone even augmented Jikes to include Parrot a some point
lucian ReachingFarr: you have both on the jvm, really
ReachingFarr: on llvm, you'd have to write them yourself 16:27
either in your language, or in some other llvm language (most likely C)
ReachingFarr plobsing: Jikes was the JVM I'm looking at. I haven't been able to find any good documentation on the internals besides a 10 year old tech report though.
lucian: Yup. That is the trade-off. 16:28
whiteknight plobsing: One thing we definitely need if we want any hope at concurrency is to decrease the use of the interp as data storage 16:30
the PCC stuff I talk about is terrible with it, storing values in the interp instead of passing them around to functions normally 16:31
That part, right there is a big killer in terms of green threads
set
set_args needs to disappear if for no other reason
plobsing yes, set_args/get_params both represent the same operation. having two ops means the operation is temporally removed from the execution of one of those ops which mandates the keeping of hidden state. 16:34
whiteknight right 16:35
and "hidden state" is miserable
plobsing I'd like to see us move to a caller-populates approach and eliminate get_params and callee-side args decoding altogether
whiteknight how does the caller know where the callee wants args put? The callee could be an HLLmapped subclass from a different universe entirely 16:36
plobsing it would also have the advantage of the caller being able to null-out dead-after-call registers before the call happens (which would allow more garbage reclamation) as well as eliminate my current TCO debacle
whiteknight: as I have it formulated right now, it is somewhat inflexible in that regard. 16:37
whiteknight Rakudo uses a custom callee-side arg binder, so nothing we do can break that 16:38
lucian in puffin, i just always pass an args array and a kwargs dict 16:39
whiteknight and then mix in MMD, where you don't even know which Sub you're going to be ultimately dispatched to, etc
lucian uh, hash
plobsing this approach would involve looking up the callee before populating the args
it splits the call-site up into more ops 16:40
whiteknight I'm really not sure I like that approach. It seems encapsulation-breakish for the caller to know about arity or other details in the callee 16:41
plobsing lookup sub; create context; populate context (previously state machine iteration over set_args signature); jump into new sub
whiteknight: one side or the other has to know how to populate
whiteknight right now, it's callee 16:42
plobsing if it is callee-side, that means the callee breaks encapsulation on the caller
whiteknight no, the barrier is the CallContext PMC
caller populates the callcontext, callee reads from it
lucian so is a CallContext a stack frame?
plobsing that's a 2-copy approach
which is 1 more copies than necessary 16:43
whiteknight lucian: right now, yes
lucian ok. so to be able to handle the call in a different memory space, you'd need to push the args in the frame
whiteknight plobsing: I like the concept. I'm having difficulties figuring out how to do it without breaking code, or breaking important guarantees 16:44
lucian pass the frame to the callee, then let go
plobsing our current callcontext is hackish to avoid the second copy by referencing the caller's frame, but that keeps the caller's frame live beyond the point where the caller has control (which leads to the residual garbage and TCO issues mentioned earlier)
lucian afaict, that's not what happens atm?
whiteknight The benefit to CallContext is that it is both the call arguments store, and the register array for the callee. So it's conceivable that the CallContext could combine stores
it doesn't change the interface with respect to the Caller if VTABLE_set_pmc_keyed_int inserts the arg directly into a register, instead of into a temporary store and then gets pulled into the register 16:45
and as far as the callee is concerned, it doesn't matter how something gets in the register, so long as it's there when we go looking for it 16:46
plobsing exactly
whiteknight in that case, I still wouldn't say the caller is placing the args, it's just building a smarter dispatch object
but maybe that's a philosophical difference
+1
plobsing now there is some magic involved for calls that don't involve a new context. NCIs would ask for a minimal context to be built and read their args directly out (as opposed to having PCC decode as is done now). Continuations would ask to populate into an existing context. 16:47
plobsing is fairly confident I have thought this through 16:48
whiteknight where we fall into trouble with this, just like my PCC refactor ideas, is in dealing with named parameters. How do we uniquely pass named parameters directly into registers without knowing how the callee wants them arranged?
named parameters are the devil
and :slurpy :named is the king
such a simple mechanism requires creating two hashes and iterating them 16:49
plobsing whiteknight: for the callee to have its registers populated, there needs to be some kind of descriptor in the callee-protocol
whiteknight plobsing: okay, so take the signature from get_params, and use that as a descriptor for the CallContext?
plobsing pretty much 16:50
whiteknight that's basically what get_params is already, and doesn't get us around fill_params
plobsing whiteknight: it gets us around *half* of fill_params - the caller side
whiteknight In that second blog post I showed you, I had timings where if we just generate the opcode stream we need to decode params on the callee side, we see performance improvements
plobsing we're always going to be walking at least one descriptor to solve impedance mismatches between callers and callees
whiteknight that's the thing, I don't think we need to walk any descriptors 16:51
the compiler knows what params we have, just output ops to fetch exactly what you need
plobsing if we use a callcontext as an intermediary, it is really just another descriptor
whiteknight but we aren't walking it 16:52
arg1 = cc[1], arg2 = cc[2]. We don't walk, we fetch what we need with constant indexes
very easy on a future JIT
plobsing next_ctx[1] = arg1, next_ctx[2] = arg2 # how is that different
? 16:53
whiteknight that's no different. That's exactly what's in my proposal
but doing both, as you've pointed out, is 2-copy passing
caller puts values into the CC using an explicit stream of PIR ops, caller pulls them out with an explicit stream of PIR ops 16:54
maybe we split the difference. positional parameters, which everybody expects to be faster anyway, can be deposited directly into registers and used immediately by a savvy callee. Named parameters require explicit fetching 16:55
if we resolve ourselves to named parameters just being slower, we can start to make some real progress
plobsing whiteknight: the problem is in who does boxing
whiteknight oh, yeah. Good point. I think callee has to do boxing because of HLL mapping 16:56
the callee can be in a different universe
I have to go to a meeting, we can chat later
16:58 theory joined 17:39 benabik joined
benabik So that's why #parrot has been so quiet today. I had closed Colloquy. 17:40
17:54 user_6374 joined 18:28 mtk joined 18:39 lucian joined 18:45 cotto joined 18:47 mtk joined 18:50 Coke joined
cotto ~~~ 18:50
19:00 lucian_ joined 19:01 rdesfo joined 19:10 Eclesia joined
Eclesia hi 19:10
plobsing o/
whiteknight is exhausted 19:18
client meetings are the worst
19:23 mtk joined 19:25 mtk joined 19:27 mtk0 joined
Coke agrees, which is why he is now in a job with very few clients, none of whom don't also work for his company. 19:28
19:28 mtk joined
whiteknight I normally interact with clients very little, which is why I'm so out of practice 19:29
but my code was being demoed today, and there's nobody better to have on hand during the demo 19:30
Eclesia whiteknight: you will see, after 3 or 4 one week training sessions as a teacher, you will find it a distracting event ;) 19:34
19:36 darbelo joined
whiteknight One thing I like about Parrot is that our users tend to be tech-savvy people 19:36
Eclesia tech-savvy ?
whiteknight Eclesia: know how to use computers
clients today wasted at least a combined man-hour trying to figure out how to open Internet Explorer 19:37
Eclesia haha, if that was not the case, they would had heard about parrot :D
never*
changing subject, I seen in the 3.7 release news, that docs where added for Archive.Tar and Zip does that mean everything to pack and unpack is working ? 19:42
whiteknight probably not
Eclesia :'(
Coke it says "docs". that's all. 19:44
not features.
Coke thought that was clear, sorry, or he would have made it more so.
Eclesia Coke: that was clear, yet having 'docs' indirectly made me think it was working 19:45
19:48 darbelo joined 20:00 mj41 joined
plobsing whiteknight: it occurs to me that callables with more complex argument massaging needs could simply use the :call_sig descriptor, as is done now by rakudo, and the result could be nearly identical to the 2-copy arg-passing you proposed. 20:00
whiteknight That's my plan for PCC, in general. kill the get_params op, and replace it with a get_callsig op, or get_context, or whatever 20:01
then when you have the callcontext, you can do normal keyed accesses to fetch values from it lazily
plobsing but that is inefficient for the simple (and common) case
whiteknight it's more efficient, according to timing numbers, than get_params 20:02
maybe not as efficient as an alternative mechanism, but faster than what we have
plobsing yes, monolithic get_params sucks
but 2-copy, always allocate an intermediary gcable, doesn't seem like an optimal solution either 20:03
whiteknight the bit problem with get_params is that it's a big loop over a descriptor signature, checking each value. Instead of seeing what we want on each iteration, we can output a single op to just get what we want directly
We need two things: A general mechanism which does what almost everybody needs, but doesn't necessarily do it the fastest, and some particular optimized hotpaths
if we know that, for instance, the call has only position parameters, we should be able to emit an optimized path 20:04
plobsing see this is where my proposal differs from that. each caller-side populator op would look in the descriptor, but it would only look at one element of it
whiteknight I'm not sure I see the difference. throw together some quick example code?
I see no reason why we would need to have only a single invocation path. We can have multiples. A standard general-purpose base mode and a variety of opt-in optimized modes 20:05
if caller and callee agree, they can use something faster, but less flexible, etc
down to something like a tailcall where they agree to just share contexts 20:06
or inlining, etc
pseudo-inlining
plobsing opt-ins require the caller and callee to agree statically, narrowing the optimization potential 20:09
(I'm working on an example) 20:10
whiteknight ok 20:11
a compiler would be able to identify and optimize those paths 20:12
but if you're loading in a library, all bets are off. You don't know how the library was compiled
or what HLL it's in, or what subclasses are mapped, or what plugins are being used, or what binder and opcode libraries are being used, etc 20:13
plobsing yes, which is why a still-fairly-general, but faster than fully general, calling convention is desirable
whiteknight I like the idea, but I'm getting trapped on the specifics
plobsing I'm going to go out on a limb and wager that most users will mostly use simple positional PMC parameters
so we should optimize that case
which is where 1-copy caller-populates comes in
whiteknight anything we do has to support tailcalls, named parameters, :slurpy and :slurpy :named, continuations, exceptions, etc 20:14
:optional and :opt_flag
The problem is, we have so many damned options that we have to support now
If I could have been here 10 years ago, I would have suggested we make far fewer feature promises to uses
users
plobsing 1-copy can support these, but the caller-side populators need to be slightly complex 20:15
whiteknight and that's another thing, any given function is going to have 1 implementation but potentially multiple call sites 20:16
adding complex code to each callsite is going to create massive bloat
plobsing but for TCO (and not leaving garbage lying around), the caller needs to be in control up until the values in its context are no longer needed 20:17
2-copy and caller-populates 1-copy both suffer the same callsite bloat issues
whiteknight the 2-copy scheme gets rid of FIA callsite descriptors. 1-copy would require them to stay 20:19
because 2-copy encodes the call information in an opcode sequence
1copy is still going to require a descriptor and a loop
I feel like the differences are going to wash out 20:20
plobsing 1-copy does not need a loop
whiteknight but still needs the descriptor FIA?
plobsing the caller-side is a sequence of ops
each op populates 1 arg from the caller ctx to the callee ctx
whiteknight I'm going to need to see that example, I'm obviously not understanding something 20:21
plobsing these ops each look at the callee descriptor to figure out *where* in the callee ctx to populate
but they each only process one arg
whiteknight so basically callee_registers[descriptior[x]] = arg_x
?
plobsing yes, in the matched case 20:23
which is the case we'd like to optimize
whiteknight Do you have anything yet? I have to pack up and leave in 2min 20:28
plobsing sorry, it's complicated to unpack all the PIR sugar
I'll have something after your commute
whiteknight okay, msg me and I'll see it tonight. Later 20:29
20:45 nbrown joined
nopaste "plobsing" at 192.168.1.3 pasted "caller-side 1-copy example" (66 lines) at nopaste.snit.ch/71673 20:46
plobsing msg whiteknight my caller-side 1-copy example is here: nopaste.snit.ch/71673 20:48
aloha OK. I'll deliver the message.
plobsing msg whiteknight in writing the example I realized that what I had said about your callsite concern was incorrect. most callsites are also returnsites, which in 2-copy means set_arg instruction sequences and get_arg instruction sequences. in 1-copy, only one of the sequences is necessary, roughly halving the bloat. 20:49
aloha OK. I'll deliver the message.
21:05 rdesfo joined 21:18 bluescreen joined 21:45 Psyche^ joined 21:47 athomason joined 22:39 AndroUser joined 23:08 rdesfo joined 23:11 Kulag joined, jsut_ joined 23:21 whiteknight joined 23:35 particle joined
whiteknight hello, #parrot 23:36
23:36 soh_cah_toa joined
whiteknight plobsing: so basically adding :descriptor metadata tags to PIR invokables, that can be used to guide optimizations? 23:43
plobsing whiteknight: I wouldn't call it guiding optimizations, but they are tags on invokables 23:55
although I suppose they could be used for optimization if the subs are statically resolved 23:58