6.2.11 released! | pugs.blogs.com | pugscode.org | pugs.kwiki.org | paste: sial.org/pbot/perl6
Set by wolverian on 12 February 2006.
00:11 jiing joined 00:13 jiing joined 00:14 feng joined, feng left 00:15 jiing__ joined 00:18 avar joined
svnbot6 r9048 | putter++ | regexp_engine_demo.pl - exploring backtracking api. 00:18
00:50 avar joined 01:04 theorbtwo joined, pdcawley joined 01:08 lao joined 01:40 orafu joined 01:51 joepurl joined 01:56 Khisanth joined 02:23 Limbic_Region joined 02:30 xern joined
svnbot6 r9049 | Darren_Duncan++ | r2519@darren-duncans-power-mac-g4: darrenduncan | 2006-02-17 18:41:13 -0800 02:43
r9049 | Darren_Duncan++ | ext/Locale-KeyedText/ : resynced KeyedText.pm version num with the perl 5 edition
02:51 justatheory joined 03:02 Limbic_Region left 03:06 hcarty joined 03:10 Cryptic_K joined 03:21 gugod joined 03:23 beppu joined 03:55 hcarty left 04:19 Amnesiac joined 04:54 justatheory joined 05:08 putter joined 05:11 justatheory joined 05:46 kanru joined
gaal azuroth: given ($object) { ... }, I think, does what you want 05:50
arcady what's the difference between { temp $_ = $object; ... } and { my $_ = $object; ... } ? 06:02
buu arcady: One involves the collapse of miniature black holes. 06:03
06:04 drbean joined
arcady oh, well then everything is perfectly clear : ) 06:04
06:04 Nouk joined 06:08 Gniij is now known as jiing 06:14 feng joined 07:14 beppu joined 07:25 scook0 joined 08:24 iblechbot joined 09:15 ghenry joined 09:24 ghenry left 10:16 Alias_ joined
Alias_ audreyt: ping? 10:17
10:18 chris2 joined 10:45 bsb joined 11:03 elmex joined 11:28 larsen joined 11:42 kisu joined 11:43 pdcawley joined 12:01 iblechbot joined 12:08 pdcawley joined 12:20 larsen_ joined 12:45 chris2 joined 12:51 avar joined 13:01 marmic joined 13:23 SamB joined 13:27 avar joined 13:34 rantanplan_ joined 13:39 rantanplan_ joined 14:36 avar joined 14:41 scw joined 14:58 avar joined 15:15 feng123 joined
avar mm 15:16
has there been any discussion on negative imports in Perl 6? 15:17
like some shorthand for:
use warnings;
no warnings 'qw';
use warnings '!qw'; or something
Alias_ that works in Perl 5... so I'm sure there's be an equivalent
unimport
avar yeah, I know it works, but I was wondering about whether there had been any discussion on a shorthand syntax 15:18
Alias_ ah
Well, the import params are module-specific
So I imagine it will be very similar
gaal import syntax isn't finalized, so i don't think unimport shortcuts can be :)
Alias_ and it isn't really unimport anyway
There's import, and then there's lexical pragma-like stuff 15:19
In P5 they are the same, I think the two were being split out or something
gaal way I see it, there's import and module config requests. there was a good discussion about this here on the channel sometime, and a few attempts on p6l, but no resolution 15:25
avar how is EXPR for EXPR parsed internally? 15:30
because it's not turned into for EXPR <BLOCK>EXPR</BLOCK>
since print $i for my $i 0 .. 5; doesn't work 15:31
Alias_ gaal: Right 15:32
gaal: imports suck anyway for most OO code, since they pollute APIs
at least in the Perl 6 symbol table style of imports anyway 15:33
err Perl 5 symbol table style
16:03 putter joined
svnbot6 r9050 | audreyt++ | * Pugs.CodeGen.YAML: "pugs -CParse-YAML" now omits the primitive 16:05
r9050 | audreyt++ | bindings for functions and types, just as "pugs -CPugs" used to do.
putter gaal: re imports, if you have any dreams of how they would ideally work, and be expressed, I'd be interested. 16:08
in case anyone is curious, my current state is: i think i'm starting to get a handle on the whole regex (and parsing, i think) picture. (ask me questions, please!:) and there's _no_ way i'm coding it all by hand. so i've just started a meta-programming exercise, so i can keep all the info in one place, and autogrovel out p5/p6 ast and engine code for p5/p6 regexs. commits when things settle down a bit more. 16:15
my surprise for yesterday was just how close p5 is to having rules already, with ast macros as subs, in the character class sublanguage (user-defined unicode properties), and string macros in the... need a name for it... top-level regex language (overload and customre). it kind of looks like the only reason p5 regex couldn't gracefully 16:19
evolve into p6 rules is the p5 engine's intolerance of reentrancy. but for that, we could have had rules years ago. sigh. 16:20
audreyt damian found that out in 2003. sigh.
audreyt mumbles something insane: "switch to pcre"
putter did anyone ever consider swapping out the regexp engine? or adding a second one (pragma'ed)? 16:21
16:22 cognominal joined
gaal putter: you were part of that discussion and I remember you had some valuable input... I'll bring this up in the 'thon 16:22
audreyt putter: yes, I think mjd and ilyaz had both coded something toward that 16:23
putter re the meta-programming, i'm trying to do a "whole spec" massage. so my hope is, when it runs (maybe a week out?), it will actually be feature complete, and perhaps even resemble correct. we'll see.
gaal: that would be great, thanks. 16:26
(hmm, so in an alternate universe, where the p5 engine was reentrant early on, "it can parse anything easily" was a property of perl 5.something, rather than p6. instead of mere Inline, complete compilers. instead of multi-year p6 discontinuity, years of moduled sourcefilter+rules-based experimentation, and incremental language evolution. sigh indeed.;) 16:34
audreyt putter: that's the theme of my CUFP article ;) 16:36
though dconway/allison remarked "perl6-rules on perl5-regex was never the plan, just a spike" 16:37
but had the p5rx engine be reentrant, the spike will had likely evolved, instead of broken on perl 5.8.3 and never picked up again
16:39 bsb joined
putter neat. no comment. agreed. putter goes to read docs/talks/cufp2005.txt . 16:40
audreyt putter: er, see docs/talks/hw2005.tex instead 16:41
putter thanks :) have read both, but rereading is fun/useful. 16:44
re dynamic grammars, turns out "once upon a time" languages with custom grammars were actually common. 16:45
died out because they were a maintenance nightmare. each file/program has it's own syntax. 16:46
I think a key characteristic / distinction from p6, is they were not (easily?) customizable enough to do other, standardized languages. eg, couldn't customize all the way to a fortran or lisp. so custom == random author whims. rather than == ability to use well designed and documented other languages. 16:48
hmm, also , the lack of a cpan / ability to share ideas, and focus in on good ones, meant "each author has his own language, with only bacterial code sharing". 16:49
"bacterial code sharing" == pick up bits of code from those you bump into, and add them to your own set. 16:50
p6 flexibility _will_ have challenges, and it will seem strange to folks, and some will remember flexibility's earlier failures, so something to think/talk about. 16:51
audreyt nod.
putter (all this because putter noticed the sentence in hw2005.tex "Unlike most languages, Perl 5 has a dynamic grammar...":) 16:52
Alias_ sigh... pcre... 16:53
putter: I seem to recall hearing smalltalk discussed like that 16:54
"It will take me 3 weeks to code up a suitable set of environment $stuff, but after that we'll progress fast"
svnbot6 r9051 | gaal++ | * DrIFT.YAML and DrIFT.RuleYAML: better error handling for unrecognized
r9051 | gaal++ | nodes in derived instances
r9051 | gaal++ | * Derive instances for the DrIFTless
Alias_ putter: And I believe you bacterial thing has a name. Something like "horozontal evolution" 16:55
your
putter Alias_: ooo, interesting. i wonder if we could get an "if only we had cpan" quote from one of the smalltalk old-timers like Alan Kay...
Alias_ Interestingly, I find JavaScript the same way 16:56
putter googles...
Alias_ Not at a grammar level, but at a "first you make your tools"
Which might help explain why Prototype is so inscrutable
:)
audreyt: A couple of very quick YAML::Syck question
?
As it related 16:57
Specifically, your Module::Install Makefile.PL for it
16:57 justatheory joined
audreyt ...yes? 16:57
Alias_ I've had 5 people in the last week ask how to do XS in M:I
So I figure it needs to be documented properly, and YAML::Syck is the only example I know of 16:58
svnbot6 r9052 | gaal++ | * Pugs.Prim - add Pugs::Internals::eval_p6y, a conjectural and
r9052 | gaal++ | probably bogosous prim to load precompiled Perl 6 files.
Alias_ Can you explain what you are doing in it briefly?
audreyt inc_paths sets the INC paths
c_file sets the C files
Alias_ why the use lib '.'?
audreyt optimize_flags sets theoptimization flags
Alias_ And why is inc_paths needed 16:59
audreyt use lib is not needed
putter Alias_: re horiz ev, yes, thanks. (as long as one stays away from creationists, which seem to have their own take on this, as on so much else;)
svnbot6 r9053 | gaal++ | * The Prelude is now precompiled into YAML.
audreyt you can take it off
Alias_ ok
audreyt inc_paths is needed becaus syck.h is in .
gaal++ gaal++ gaal++ # 3-day non-stop hacking
Alias_ So it refers to the location of the headers? Because the header search is done in the @INC?
gaal the next one's the nice one :) 17:00
audreyt Alias_: it's not done in the @INC
putter "bogosous" *grin*
audreyt inc_paths refers to C's "INC"
not perl's @INC
Alias_ uuum, oh dear
audreyt I admit it to be ambiguous.
c_inc_paths?
header_paths?
Alias_ I have some ideas, I'll deal
putter gaal++ woot
audreyt k
Alias_ (and leave the current one for now)
gaal audreyt++ audreyt++ audreyt++ # nonstop mental-and-I-mean-mental support :)
Alias_ can_cc I've already improved 17:01
audreyt mm I'm feeling mental
cool
Alias_ explain c_files
gaal whoops, "next one" was actually 9053
Alias_ According to someone else, it's a bit weird
Because you set c_files( 'foo.c' ) if there's a file 'foo.xs'
Or something
gaal plus profound hackerosity magic and generic^Wall-around wonderosity coolage, etc. :) 17:02
putter lol; audreyt++ ;)
gaal so, now, just to make it not slow, TWD etc.
audreyt Alias_: I think c_files should accept .xs and translate them into .c 17:03
putter so this putter around the planet thing seems to work.. ;)
17:03 Maddingue joined
gaal remakes an optimized build. 17:03
audreyt c_files glob("*.c"), glob("*.xs")
putter err, audreyt-ing around the planet...
audreyt putter: I'm thinking about relaxing my self-imposed US ban
just so I can come to Chicago and Portland.
and Seattle.
(and where stevan is at, now that he did not secure a TPF grant to .jp) 17:04
where is stevan located anyway?
putter avoids flying where possible. security idiocy was the only thing i didn't like about flying _before_ 2001. now it's just silly.
Alias_ Entering the US sucked 17:05
audreyt it looks like YAPC::NA this year will rock, though.
Alias_ And I'm from a friendly country dammit
audreyt: I may not be going :/
putter stevan is in Conneticut. ok, no chance of spelling that right without checking. a couple of hours from boston.
audreyt yeah, I'm from a minion country too
ah, on the other end of the country 17:06
gaal putter: consider donating to alt.pave.the.earth so you can take a train anywhere!
Alias_ audreyt: World Cup is on at the same time as YAPC::NA and AU is going for the first time in 35 years
audreyt Alias_: that means... what?
(and world cup for which sport?)
Alias_ err "the" world cup
football
putter lol
Alias_ That means how could I go somewhere where I can't find room of drunk cheering australians!
putter didn't know the NFL had a world cup... ;) 17:07
Alias_ heh
If YAPC::NA can organise drunk rooms of cheering australians, I'm there
or a football mini-conf :)
Anyways... c_files 17:08
putter seems likely :)
Alias_ Is there any reason you could just want it automated :)
compile_all_c_files
or line it into the inc_paths
putter back to meta-coding...
Alias_ compile_c_path '.'
audreyt Alias_: maybe... but it's unusal to put all .c into the same . 17:09
Alias_ wouldn't it be recursive?
or is that not normal too
I'm kind of feeling around for the "as easy to learn as Ruby" approach
(which is my new MI slogan)
:)
audreyt recursive may make sense 17:10
Alias_ so people would split it up, but #include "foo/bar.h" ? 17:11
I've never really understood C namespacing/dirs/etc
audreyt the thing is everyone does it differently 17:12
see Compress::Zlib for another example
Alias_ got any more?
Even finding YAML::Syck was hard
audreyt the usual case 17:13
where it's just a .xs file
requires no special declaration whatsoever
it's only needed if you either include other .h/.c files
Alias_ meaning one and one only .xs file in the root dir, or something like that right?
audreyt yup 17:15
Alias_ Is that a EUMM thing, or can we remove that? 17:17
It would be nice to keep things declarative an d predictable, rather than magic :)
audreyt it's a EUMM thing, though I'd argue it should Just Work much like .pm files 17:18
Alias_ nods
audreyt (I may need to run for dinner soonish)
Alias_ cc/stuff
or something
So, something like cc_autocompile '.'
although that reads wrong
cc_build '.' ? 17:19
which would be cc_inc_paths + File::Find and add all .c and .xs
audreyt that sounds good
Alias_ And I'd like to rename c_files to cc_c_files, but it's not as important as inc_path 17:20
it also looks wrong :)
17:27 pdcawley joined, calanya joined 17:28 kisu joined 17:52 justatheory joined
rafl audreyt: Do you already know if you can come to GPW? 17:52
Alias_ hmm, I just realised an amazing feature of Module::Install 18:01
It allows you to deprecate old features really really quickly :)
(in relative terms)
18:04 Amnesiac joined
putter Alias_: how so? 18:08
Alias_ Well, because it doesn't matter what the end user has, and only what the author has, it means a relatively much smaller number of people need to adapt 18:09
And authors are generally much smarter and easier to control 18:10
putter ah :)
Alias_ So if I replace c_files with a die "Deprecated, use cc_c_file" I can skip the whole "keep it" and "keep it, but warn" stages
putter got it. a nice property. 18:15
wonder how more things could be made to have it...?
Alias_ heh
not many
I suspect it's a particular property of installers
Bundling redundant extra code with you is a positive in that case
putter hmm.. though in p6, you can request particular flavors of module... i don't remember if foobar-joe-1.30 and foobar-bob-0.05 modules can coexist at runtime... i think so... 18:19
Alias_ yes
C/C# style versioning
obra putter: larry put this in the synopsis. 18:20
putter err, "C style versioning"? ;)
obra: ah, great. tnx
hmm. foobar-1.3.0-joe. different order, hmm... I imagine one might end up saying a lot of use Something-(Any)-joe. because joe and bob have entirely unconnected concepts of what the name "Something" applies to. the hierarchy looks like authority-name-version, but it's been shuffled to optimize for the "agreement on name" case. 18:26
18:47 orafu joined
putter It could be nice to have an "annotated" p6 spec. which not only said what's what, but also described the thinking behind it. maybe? 18:49
Alias_ audreyt: ping? 18:50
18:51 nnunley joined 18:54 ilogger2 joined
robkinyon I'm not sure where to ask this, but what is "self-tying" and why is it so horrible? 19:10
I ran into the "self-tying of arrays and hashes is not allowed" error and the only thing i can find on it through google is that it was removed in 5.8.9 19:11
errr ... 5.8.0
(I'd ask in #perl, but I've asked questions there before)
putter robkinyon: re what is, just from the name I imagine it's something like {package A;sub new {bless $_[1],$_[0]}} my @a; tie @a, A, \@a; so @a is both the variable tied, and the object it is tied too. aside from being hard to use (A methods would have to untie/retie if they wished to use its contents), perhaps it required 19:55
19:55 qwacky joined
putter some internals gymnastics. ie, @a, the object itself (not merely its contents), would have a ref to itself (the tie ref), so the gc would have to understand this possiblity, and untie it if it had only one ref... so perhaps pain and limited utility. 19:56
stevan audreyt: ping 20:06
hey putter :)
robkinyon putter: Thanks 20:09
i don't think it would be that hard to use 20:10
but, i was storing object data in a separate %object_data hash keyed by the stringification of the retval of tie( 20:11
audreyt stevan: yes?
stevan: I'm 1.5 chapters into TAOMP 20:12
stevan audreyt: I havent heard back from tpf yet, do you know something I dont ? :)
audreyt stevan: I happen to, yeah :)
stevan ah,.. oh well
how do you like AMOP? 20:13
audreyt so I'll come visit you instead
stevan audreyt: that works :)
audreyt AMOP is nice
stevan however, the sushi is not as good here :)
audreyt though very elementary so far
because you've brainwashed me already
stevan audreyt: :)
the more interesting bits come when they break down the ways to extend the MOP 20:14
I am working on that kind of stuff with Class::MOP right now actually
I have been thinking that we need to seperate the MOP for p6 from the new/bless/CREATE/BUILDALL/BUILD part 20:15
robkinyon stevan: Will the MOP allow you to specify how WALK and WALKMETH is going to work?
stevan robkinyon: I think it should yes
robkinyon That would be very cool. 20:16
stevan in other words,.. I dont want the MOP to be tied to S12 at all
audreyt <aol/>
I think that's the way to go, yeah
robkinyon does that mean you want to describe S12 in terms of the MOP or are S12 and the MOP in parallel? 20:17
stevan S12 should be implemented in terms of the MOP
20:17 KingDiamond joined
stevan I am gonna try doing it with Class::MOP first 20:17
robkinyon So, if that's the case, then can I ditch S12 and implement Rob::S12 instead?
stevan see what happens there 20:18
robkinyon s/can I/should I be able to/
stevan robkinyon: yes you can :)
robkinyon LOL
stevan S12 ties the MOP down in too many places and makes things like repr types and containers a real pain
they should not be a pain, they should be simple, but they should not be tied to the rest of the object system 20:19
20:20 buu joined
robkinyon So, you're saying that S12 should make a set of duck-typed requirements upon the thing that I can build with the MOP 20:20
then, if I want, I can build Rob::S12 and someone else can build Ruby::S12 and everyone's happy?
stevan no, I am saying that there should be a MOP deep down which gives you the most generic (class|prototype) based object system we can muster, and what you hack on top of that is your business :) 20:21
S12 will be the default 20:22
to a certain degree the MOP becomes an implementation detail
robkinyon but, there are going to be P6 subsystems that are going to depend upon a certain implementation of containers
stevan which allows parts/all of it to be optimized out if you want
robkinyon: nope, same deal
implementation detail 20:23
robkinyon uhh ... i don't think so
stevan everything should depend on the public APIs
or its an implementation detail IMO
robkinyon which are built atop of S12 20:24
or some S12-implementing item
robkinyon & # dishes ... groceries .... you'd think I was married!
stevan S12 is a public API
things must conform to it
20:27 orafu joined 20:29 sapper joined 20:33 p5evalbot joined
svnbot6 r9054 | audreyt++ | * pugs -CParse-HsYaml backend for yaml-node based serialization. 20:48
r9054 | audreyt++ | * slightly improved DrIFT.YAML syntax.
Alias_ What's DrIFT do btw? 20:51
A YAML implementation in Haskell/P6/PIL? 20:52
gaal no, it's a haskell-intelligent preprocessor. it lets you do metaprogramming in haskell.
Alias_ That sounds scary... 20:53
gaal we use it to add YAML serialization instances to our data structures
Alias_ So now Haskell is even more obfuscated :)
gaal ("instances" in haskell means something like "role consumer")
Alias_ again in english? 20:54
gaal nah, consider this example.
you have a run of the mill data structure that you defined
data Color = Infrared | Red | Orange | Yellow | Violet 20:55
in haskell, you can add a trait to this type, saying that these possible variants are equatable
so that haskell automatically knows to tell that Orange == Orange but Red /= Orange 20:56
this is spelled simply "derives Eq"
Alias_ is /= is !=?
gaal the nice thing is that unless you're doing funky stuff, you don't have to suply the code for this yourlsef
Alias_ erm, is /= haskell for != ?
gaal yes
integral =^H/ 20:57
Alias_ ok
And so this will let you bulk-import stuff from yaml files?
gaal you can similarly say that these different possible values have an inherent ordering
Alias_ right
gaal so you want to ask whether Red < Infrared and get False 20:58
you can do this with "deriving Ord"
(s/derives/deriving/ above)
that's a standard "typeclass"
Alias_ deriving being in essense "dwim as long as it's right"
gaal in Perl 6 Roles fulfil the same, er, role, more or less 20:59
Alias_ to the parser...
right
gaal you can say that your data type does (say) Eq, but doesn't inherir the defualt implementation at all
there are a number of useful standard classes 21:00
Alias_ right
gaal two very cool ones being "Show" and "Read"
which stringify stuff and back
Alias_ And so when do we turn to what drift does :)
gaal ?eval {a=>"moose"}.perl
ok ok :)
21:01 evalbot_9024 is now known as evalbot_9054
evalbot_9054 "\{(\"a\" => \"moose\"),}" 21:01
gaal now say you invent a new typeclass of your own
oh, I don't know, YAML :)_
you don't want to go to your 17000 user-defined types and write code ("methods") for them, right?
the computer can do that for you 21:02
Alias_ do you normally do that though
gaal but it's not so simple to let something like the c preprocessor do it, because sometimes you need to be very careful
Alias_ Do most Haskell programmers have big chunks of code for types?
gaal how do you mean?
Alias_ Are types done currently with code 21:03
Is this filling a need in that way
gaal oh, well, by type we mean a type of a data structure 21:05
Alias_ asin Color
as in
gaal so a node in the AST can be say a function application
Alias_ erk...
gaal App Exp (Maybe Exp) [Exp] 21:06
Alias_ OK! My LambdaCamel shipped without a Camel!
gaal don't worry, that simply means...
Alias_ This Camel needs a Camel :)
gaal say "hi!"
Alias_ don't worry 21:07
I just spent 4 hours debugging CPAN.pm on the new Win32 Perl, my WPU is nearly spent
gaal ==> _Apply_ the code _&say_ to (no invocant, this is a function) with the arglist ["hi!"]
if you enter the interactive pugs shell and do :d say "hi!", you'll get it pretty-printed in one format 21:08
because App is an instance of Show 21:09
Alias_ ok
gaal (well actually, App is just one variant of an AST node, like Orange is just one variant of Color)
so any Color knows how to Show itself
as does any Exp 21:10
Alias_ runs
21:10 Odin-FOO joined
gaal if you want to have a Color show itself in YAML, you have to write an "asYAML" method on the type Color 21:10
21:11 Odin-FOO is now known as Odin-LAP
gaal that would emit the part of the YAML representation that is needed 21:11
avar does p6 yaml support anon functions?
gaal in the case of a color, it's trivial, just a string
avar: sure 21:12
Alias_ gaal: wouldn't that open the door for major security issues?
avar $ perl -MYAML -e 'print Dump $s = { subptr => sub { "foo" }}'
---
subptr: !perl/code: '{ "DUMMY" }
p5 doesn't...
gaal: is that compatable across languages?
i.e. can I run my perl 6 anon subs on ruby
Alias_ avar: YAML isn't compatible across languages
gaal in the case of sometihng more complex, like the syntax for function application, you have to stanradrize on a representation you can make sense of when you want to deserialize 21:13
avar it kind-of-is
Alias_ avar: You need to have a matching serialize/deserializers for each thing in each language
avar: Otherwise... erm... something
gaal avar: grab a recent pugs and do "./pugs -CParse-YAML -e '{ say "I am a closure" }.()' 21:14
how's this a security issue?
Alias_ gaal: The whole point of most exploits is to trick you into running code
gaal: When your data structure embeds arbitrary code, that gets a whole lot easier
That whole data/code seperation issue 21:15
integral but it's not encoding code, it's encoding data...
gaal now it turns out that for the majority of data types, there's a standard way of doing things that works
but sometimes it breaks down, notably in circular references
integral it's not code code, but code as an AST, reified, so it's not actual code when it's in the YAML
Alias_ integral: um... { system 'rm -rf ~' }.()
it still does things 21:16
integral Alias_: no, it's not like that at all. It's like giving you the string "rm -rf", rather than an executable shell script of the same
gaal Alias_: it's no different from the risk of eval
Alias_ gaal: Which is why Data::Dumper is completely unsafe as a data serialization language
And why we use things like XML
integral Alias_: umm, my point is that when this YAML is unserialised this stuff isn't run...
Alias_ integral: are you sure? 21:17
gaal NAK. we are serializing code, not business data
Alias_ What if it's a init code to a class hook that runs it
gaal the whole intention is to run this code
integral yes, it's the difference in Data::Dumper output between: $VAR1 = { case1 => "sub foo { unlink('.') }", case2 => unlink(".") };
gaal if the serialized stream is tampered with, sure an attack can be injected
integral Alias_: we've got the first, because this code *is* data. Because we're INSIDE a compiler.
gaal that's like me editing your files on your fisk.
*disk. 21:18
Alias_ integral: Right, and the exploit would be finding some deserialisation class that runs it (though you don't realise it might)
or finding some other way to trick you into running it
integral Alias_: A deserialisation class that just eval'ed any string?
Alias_ integral: See, you hadn't even considered it
integral I think someone's either being silly, or misunderstanding how to exploid serialised things. 21:19
gaal Alias_: we're talking about things that are all replacements for "eval" in use anyway
integral *exploit
Alias_ someone goes and uses this shiny new anon code feature, and it only takes one person to use it in some bit somewhere and forget it's completely untrusted
integral umm, no. 21:20
Alias_ And if you have that things installed, all of a sudden I've got arbitrary execution
gaal whoa whoa :) what shiny new non code feature? :)
integral Alias_: Let's imagine two readline functions.
Alias_ or perhaps closer, a Storable class
integral One reads a line from the keyboard and returns it. The other does that, but if the first char is ! interprets it as code, and returns the result of that code.
Data::Dumper has the second feature, YAML does NOT.
Alias_ sure it does
Because it has equivalents of Storable hooks 21:21
integral no, it doesn't have something which says "instead of serialised data, here's some code instead that will give you the data"
Alias_: but they're not in the data stream itself.
Alias_ right
integral Someone would have to put a hook in the app that did an eval. But *all* serialisation schemes can have someone put an eval into the app! 21:22
Alias_ not the app
gaal Alias_: the purpose of YAML in the internals work we've been doing is not user tools (although those could be useful)
Alias_ Any class
Any class that acts as a YAML hook
Because it will autoload the class
or at least, if ingy did something similar to Storable it well
will
(because I don't know any other way to do it)
integral ugh, autoloading classes?!
Alias_ How else do you deserialize things 21:23
gaal which things?
Alias_ oh, for example, a singleton
or a database handle
integral bleh, I like haskell too much where "data" is this nice fixed well defined thing, and it's just typeclasses that vary
gaal and for which purpose?
Alias_ gaal: To serialize a larger object that contains one
gaal Alias_: for the compiler, of for the application? 21:24
integral anyway, this particular "feature" of the perl YAML has nothing to do with the haskell YAML luckily.
Alias_ integral: It's not a feature of Perl so much as a feature of any sufficiently advanced serialization mechanism
It requires some sort of code-assisted serialize/deserialize
integral Alias_: and an open world model. 21:25
Alias_ And so what happens if you don't have every single class already loaded you encounter in the data stream?
it dies?
I hope so
gaal are you talking haskell type or perl class?
Alias_ language-neutral
integral alias must be talking about perl 21:26
Alias_ deserialization handler
gaal don't confuse the internals of the compiler with user-visible technologies perl provides
Alias_ YAML has class-tagged nodes 21:27
gaal sure
Alias_ they have handlers
?
integral sure, but in a closed-world application all the handlers are part of the app.
Alias_ ok 21:28
So you start a YAML stream
integral and the current YAML for haskell has no idea of how to start using hs-plugins
Alias_ And you're going along, and YAML says node "Foo"
And you don't have a class Foo loaded
then what? exception?
gaal who's "you"? a perl program or the perl compiler?
integral yes.
The thing haskell is loading has a concrete type, not just an interface that it complies to. 21:29
Alias_ ok then
So no way to load that class
that's all fine
21:29 dduncan joined
integral We know it's *exactly* a tree made from these types of components, not the other view where we know it's a "tree" that has all these methods 21:29
Alias_ it introduces other problem, but those are functionality thigns
uh... a yaml tree? 21:30
oh, haskell
I wasn't talking about any particular language
It's not a language issue, it's a serialize/deserialize algorithm problem
integral *sigh*
gaal Alias_: I still think you're confused. The current YAML work is, in the bottom line, just one technique for the compiler to store program data. 21:31
Alias_ And you gave me the answer I wanted, in Haskell's case it can't handle encountering a class-tagged node that isn't already loaded
gaal it's program data anyway. if someone could manipulate it they could make the perl it represents do evil things
integral Alias_: that's not a very good way to be viewing what you've got.
gaal but I don't see your threat model.
integral In haskell you've got data, plain old data, it doesn't have methods. What you normally seralise in perl are objects and an object is both data *and* the behaviour. 21:32
Alias_ integral: That is not the issue
integral Yes, it is.
Alias_ no, it isn't
integral Yes, it is.
Alias_ The threat model NEVER has to call an object method 21:33
So the object's behaviour is NOT a problem
integral But if you're not loading an object, there is no "class tagged node that isn't already loaded"
Alias_ right
well, not entirely, but close enough
integral That's my point, your final statement was close enough but not entirely. 21:34
Alias_ but lets say yes for the sake of argument
gaal Alias_... we're currently loading nodes in a parse tree. If someone controls these nodes, the program that'll get run will be pwned anyway
Alias_ That's not true... 21:35
Otherwise XML would be impossible
And you could exploit everything
gaal well anymoose. I have to sleep now
good night :)
21:36 calanya joined
integral Alias_: well you can. XML that is XHTML containing a <script> tag that is run with full permissions of the browser user. 21:36
Alias_ exactly
because someone OTHER than you implemented it
gaal zZ&
integral So, yes, in this case, the one gaal talks about, it is true.
Alias_ But in that case, you know it's XHTML 21:37
actually, it applies
ideally you'd be using data types with no support for that
So you can not have to worry
And no support for injecting arbitrary types into the node stream 21:38
because the format is closed
YAML supports arbitrarily tagged nodes
Much like I guess you could say any XML document could set a namespace to be XHTML 21:39
integral gives up &
Alias_ If the XHTML handler loads on demand, then just by virtue of having an XHTML handler installed, any XML document can execute arbitrary code
21:39 DesreveR joined
Alias_ without ever calling a "method" on the <script> object 21:40
See, I said I needed a camel 21:42
... with my lamda
Alias_ wander off
audreyt Alias_: YAML has arbitrarily tagged nodes, yes. we ignore everything but the tags we know how to handle. 21:45
Alias_ See!
One camel and my issue is resolved in a single line
audreyt it's a fatal error, actually.
Alias_ :)
good
Then it's not a security problem
audreyt I'd be very sure it's not, yeah :) 21:46
Alias_ Well, unless you preload any handler you don't understand that executes the code block
either way, I still think it turns something patently stupid "executing an arbitrary string of code" into something more tempting "executing nice trustworthy code trees"
audreyt yes. 21:47
though we were doing that patently stupid thing at the first place
in loading Prelude.pm and run it
Alias_ and that approachability is often half the problem
audreyt and we are just replacing that into the more tempting (and easier to check) code trees.
Alias_ well, it's the handler that's the problem really, not the actual code
audreyt right. and in this case, we write no handler ourselves 21:48
Alias_ that only matter once the handler might potentially execute it
audreyt the handler is derived from the static structure of the types that we are accepting.
there is no human intervention possible in writing handlers.
Alias_ in your case :)
audreyt yeah. :)
Alias_ For an exploit to happen, generally 5 things go wrong
audreyt (I agree with your general analysis.)
Alias_ 1 being "YAML supports code"
audreyt (just that we are not tripping it) 21:49
Alias_ right
BTW, I'm very close to throwing Module::Install::Compiler away :)
audreyt excellent
Alias_ I had a read of what the makefile params actually do
And I'm amazed you can make anything work with what's in ::Compiler :)
well, not true 21:50
I'm amazed you can make something like YAML::Syck work
audreyt excellent :)
Alias_ Which would seem at first glance to exceed the notional complexity threshhold
I'll probably need some help here and there to work out which commands can be implemented that are useful to real life XS coding 21:51
I have some ideas
but I'm dealing with some support code first
I also don't have the time for a job this big :) 21:52
But so you know, don't release 0.57 without talking to me first
audreyt I'll leave it for you to release anyway.
Alias_ Because there's some transitional things there and it's not really quite in release state
I'd also like some help at some point on testing 21:53
Create me a sample test case for example...
not sure exactly how to make one without risking actually installing things or interacting with the parent install
:)
audreyt *nod* 21:55
Alias_ And I'm trying to resist rewriting the whole thing
But that's par for the course
:) 21:56
And it's fractally complex, so too many risks
21:59 ayrnieu joined
svnbot6 r9055 | gaal++ | * DrIFT.YAML - add a default, erroring instance for fromYAMLElem, 22:43
r9055 | gaal++ | thus eliminating trillions of pesky warnings.
gaal you can tell you're under an Audrey field when you commit after you'd gone to sleep 22:45
beppu Alias_, this is the Module::Install ticket you asked me to write last night: rt.cpan.org/Public/Bug/Display.html?id=17731
It's about being able to customize what happens during installation.
Alias_ ok 22:46
23:32 scook0 joined 23:35 drbean joined 23:51 lumi joined
Alias_ beppu: I've replied to your bug btw, on the subject for share, which does exist 23:51
23:51 gaal joined
beppu I'll take a look 23:52
Alias_ beppu, and we've got limited etc capability
See...
File::UserConfig
That's a user-specific etc, but it fits into the resource model properly 23:53
beppu I didn't know about these.
Alias_ I only wrote them at Christmas :)
The share stuff has existed for ages, but is arcane and few people know about 23:54
I wrapped some user-friendly shell around it when audreyt told me it existed
The File::UserConfig is just a combination of that, plus the new File::HomeDir implementation
more an elegant trick than a full on resource though 23:55
but correct
beppu I'll consider using these.
Alias_ doing system etc has issues
beppu It's hard to be portable, right?
Alias_ So I have no answers for that, yet
very
Perl runs on 100+ platforms
which is under the skin, share is a variation on lib 23:56
which is why ...
And homedir is only tested on Win32, basic Unix and OS X
So there might be variations
beppu that covers a lot, though.
Alias_ enough for desktop applications :) 23:57
Which is what I'm using it for
beppu It's mostly Win32 that would be problematic, I would think.
Alias_ it's all three
for homedir at least
Mac OS X is not entirely like Unix
beppu what about the share stuff. How does that work using File::ShareDir? 23:58
nvmd. i rtfm'd. 23:59
Alias_ Module::Install::Share provides the stuff to install it to the right place, which is the same technique used by autosplit and other core modules
File::ShareDir adds the logic to locate it afterwards

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!