6macros: discussing the finer points of Perl 6 macros, Qtrees, and how to stay sane | irclog: irclog.perlgeek.de/6macros/today
Set by moderator on 28 July 2015.
00:17 Ven_ joined 01:21 Ven_ joined 03:35 Ven_ joined 04:36 Ven_ joined 05:38 Ven_ joined 05:41 vendethiel joined 06:48 Ven_ joined 08:36 pdcawley joined
pdcawley Morning _Ven 08:38
09:51 Ven_ joined
Ven_ ahoy 09:59
did I link eslisp here already? 11:10
yes I did
pdcawley _Ven: If we don't have an explicit quote form, doesn't that make (the relative common) elisp pattern of: `(func-name (quote ,symbol) ,@args) a wee bit tricky to manage? 11:30
Ven_ we don't use symbols for names 11:31
barring the hygiene problem for now:
quasi { sub {{{$name @ Q::Identifier}}}({{{@args :: Q::ArgList}}}) {} }
pdcawley So, how do you arrange to expand a macro into a function call which accepts an expression in quoted form as its argument? 11:32
Ven_: That sets the type of the unquoted thing? 11:33
Ven_ supposedly, yes :)
quasi is quote, {{{ }}} is unquote 11:34
pdcawley And 'quasi { ... quasi { {{{ when_is_this_unquoted }}} }'?
Pretend I balanced up the } at the end correctly. 11:35
Ven_ the unquote-time is something masak blogged about already 11:36
I can't remember the name he used
but he seems he wants them macro-late-bound (i.e. replaced when the quasi "escapes" the macro iirc)
pdcawley I'm not sure I get where he's going with that tbh. 11:37
macro expansions and quasi expansions surely all happen at parse time.
macro ...: turns ... into (say) a quasi quote which can only 'see' the scope of the macro body that generated it, macro returns a fully expanded syntax tree at that point, and the parser continues on its way. 11:39
If you need to capture something for use at runtime, you do it through the standard mechanisms of closures, etc.
things get icky if you don't have a way to introduce 'anonymous' identifiers that aren't going to accidentally shadow anything in the containing scopes. 11:41
Ven_ agrees ;)
macros are hygienic in 007, afaik
so no gensym problems
pdcawley if you can't do macro foo($expr) { $memo = gensym; quasi { my {{{$memo}}} = {{{$expr}}}; ... } } 11:43
then you're in for a world of pain.
And if you insist on hygienic macros, you're going to have to understand far more about Scheme's hygienic macro systems than I could ever be bothered to do.
How implemented are macros? 11:45
Ven_ dunno 11:52
also, just like I said, `my $memo` inside the quasi would work fine right now
because macro have a specific lexpad (Perl 6 talks about a COMPILING namespace as well)
there are always way to work "around" hygiene if you need to. it's not like you can't write anaphoric macros in scheme 11:53
racket offers a few more ways around (syntax parameters, etc)
pdcawley Ven_: problem I have with the various 'ways around' is that the basic kit of parts ends up being more complicated in cases where you might not want them to be. 12:02
Explicit gensym can be a little like juggling hand grenades, but it has the great advantage of the language not deciding for you what the Right Way is. 12:03
Ven_ sure 12:04
but sometimes, it's good the language takes a safer default
and gives you enough tool to disable that default where needed 12:05
i.e. racket's format-id (in racket/syntax)
which I used, say, github.com/vendethiel/brain.rkt/bl...rs.rkt#L35
masak re hygiene -- you should be able to do COMPILING::{{{$name @ Q::Identifier}}} 12:10
pdcawley needs to go and read up on this, methinks. 12:11
Hmm... consider: macro foo ($expr) { quasi { { my $bibble = '22'; {{{ $expr }}} } } 12:16
camelia just fell over in a heap when I did 'quasi { $bibble }.PERL.say' because it couldn't see a declared $bibble. Thought (one of) the point(s) of quasi was to just make an AST without worrying to much about scoping rules. 12:17
masak who told you that? :) 12:20
pdcawley Every other language that has quasiquoting. 12:23
Quoted form is entirely unattached to a scope, unquoting within the quasi happens in the containing environment, but the quoted form doesn't have a scope attached until it's plumbed back into some evaluable context. 12:24
masak maybe this discussion will be easier if we both start from what's in S06...?
I mean, much of it is superseded or obviated by things I've discovered lately, but the hygiene and scoping is pretty much in there and pretty clear. 12:25
pdcawley masak: can I write (say, and I know that perl6 has builtin ways of doing this, but bear with me) a macro like: anaphoric-map { $it * 2 } @args; where the 'anaphoric-map' expands to: map -> $it { $it * 2 } @args, or map -> $it { $expanded_expression_involving_it } @args? Because if that's not possible without tapdancing on the calling side of things, that rules out a whole class of useful macros. 12:29
masak yes, I know. I've had basically the same discussion with Ven. 12:30
I... I don't know yet.
the way things parse in Perl 6, you'd have to at least intercept the macro before it gets to that block and sees $it 12:31
it's possible 'is parsed' might help with that, but I don't feel things have clicked with 'is parsed' quite yet. 12:33
pdcawley masak: Point is, if you spot a macro during parse, then all bets are off as to the environment of the rest of the parse. Have to wait 'til the macro grabs the expressions it needs (parsed in a 'lax' environment, allowing undeclared variables to get through) and returns an AST, only then can you turn stricture back on.
masak we've had to give up and do that in quasis already. 12:34
because quasis can contain unquotes, and you can't apply the macro anyway until the unquotes are "filled".
pdcawley parsing the 'arguments' to macros as if their scope is already known is a recipe for failure. 12:35
you mean i'm going to have to quasi quote macro _arguments_? That's insane.
masak no, that doesn't make sense. that's a category error. 12:37
pdcawley Okay. Misunderstood you then.
masak I'm sure we can work things out little by little :) 12:38
I am eager to bring macros to Perl 6 in a way that makes sense. this means listening to Lisp advice, but also not heeding it without understanding the consequences, and the different forces that apply to the Perl 6 design. 12:40
pdcawley Hmm... just thought... say you have a noddy macro that takes an expression that contains a variable name as an argument. When you do: quasi { {{{$arg}}} }, then the name in $arg is out of scope when the quasi is built. No different from quasi { $out_of_scope_var }, so rakudo as she stands is going to barf horribly at that point. 12:41
12:41 Ven_ joined
masak pdcawley: currently the rule is this: quasi blocks respect lexical scoping. synthetic Qtrees (created using .new on Q types) don't have any such limitations. 12:42
you can also "detach" a quasi or macro argument from its scope by calling .detach on it
but of course, by the it might be far too late, if you already got a parse error 12:43
pdcawley That seems arse about face.
masak so maybe what Ven++ said makes sense, and it should also be an adverb...
pdcawley Once a quasi has finished unquoting its contexts, it should be entirely detached.
until it's plumbed back into the ast at the macro call site. 12:44
masak but the unquoting doesn't happen until sometime in the middle of the macro call, quite a bit later than the unquote is parsed 12:45
still at compile time, but later during the compile
Ven_ irclog.perlgeek.de/6macros/2015-11-03#i_11474298
pdcawley masak: No.
Ven_ I talked about racket's syntax parameters for that, and a few other solutions
pdcawley Unquoting happens within the scope of the macro call. Macro returns a clean AST.
Surely.
Ven_ > at least intercept the macro before it gets to that block and sees $it 12:46
masak both of you seem to have pretty wild disagreements with actual Perl 6 spec of how Perl 6 macros should work...
pdcawley Since the actual perl 6 spec specifies that Perl 6 macros do impossible things (like be callable at runtime unshielded by an EVAL), then I'm taking it with a serious pinch of salt. 12:47
masak I may be able to afford such disagreements, but only if I happen to stumble over a *fully-formed* proposal of how it *should* work within the bounds of Perl 6.
yes, please feel free to take it with a pinch of salt. I do.
but it's pretty clear on the lexical scoping.
and I haven't really seen a better proposal, only Lisp-inspired dreaming.
pdcawley masak: I'm going by the idea that a macro returns a 'syntax tree that needs no further parsing'. Since parsing seems to be intimately tied in with scoping, I'm really not sure how that's possible. 12:48
masak ok, let's make this very concrete
github.com/masak/007/blob/master/t....t#L46-L59 12:49
as of yesterday, that test there passes
pdcawley masak: That should be a bug.
Only way you should be able to get at x is through an unquote. 12:50
quasi { say({{{x}}}) }
masak in Perl 6, macros have hygiene.
if you don't want the hygiene, use COMPILING:: 12:51
pdcawley macro foo(varname) { return quasi { say({{{varname}}}) } } <- how is that different? 12:52
Ven_ pdcawley: yeah, I argued the same 12:53
pdcawley: : because varname is an AST there
pdcawley You're just complicating things horrifically.
Ven_ not a value
but I agree "x" shouldn't work. that's not the right time.
masak I'm sorry, but you're both tilting at windmills at this point 12:54
Ven_ "x" works there because it's just a value. {{{ }}} is not about the evaluation time, it's about interpreting its content as an AST
masak because I *do* understand how that works. I've implemented it. twice.
Ven_ oh, I know how that works. I just explained it 12:55
masak oh, good.
yes, and I think I agree with your explanation.
in the way "x" works there, the quasi is basically just a normal block. 12:56
pdcawley shudders
masak duly noted. next topic? :) 12:57
pdcawley So... if I know (or hope) there's an x in the outer scope, I do quasi { COMPILING::x }?
masak right
and *that* one gets late-bound and checked at insertion, like you want
pdcawley And, say I have my $foo = 10; macrocall $foo; and I want that to expand to: 'my $foo = 10; do { my $foo = $foo + 1; say $foo }', how do I write the quasi. 13:00
Ven_ masak: so, my example in #perl6, is it working in 007?
pdcawley afk: lunch 13:01
Ven_: Hadn't even thought about state...
Ven_ well, I did, so I'm concerned ;-) 13:02
masak Ven_: 007 doesn't really have state. 13:06
pdcawley back 13:59
Just a thought, the problem I'm having grokking quasi as it stands in perl 6 is that it isn't really quasiquote. 14:00
If you were to make it so that everything in a quasi block had to be explicitly unquoted, named in full or actually declared within the quasi I'd be less bemused by it. 14:01
The magic interpolation of variables without explicit unquoting seriously fucks with my head.
So, the test that masak posted should be a syntax error (because x is neither declared in the block nor fully qualified). I'd expect {{{ $not_an_expr }}} to be equivalent to {{{ $not_an_expr.perl.AST }}} or whatever the magic is. 14:04
masak since I've already been over most of this with Ven, I have a comment from an issue ready: github.com/masak/007/issues/63#iss...-152511022 14:06
if you could give your thoughts on that comment, I'd much appreciate it 14:07
pdcawley If you're not returning a fully resolved AST from the macro, you're going to be making a rod for your own back. 14:12
masak let's come at this from another angle
in Perl 6, parsing is very much intertwined with symbol lookup
pdcawley Then that basically makes macros impossible.
masak please listen for a bit
pdcawley masak: Just looking at your 'return'.
masak is `foo bar;` a valid expression or a syntax error? it's the former if `foo` is a function. it's the latter if `foo` is a type.
pdcawley and you can't know if foo is a macro.
Because you don't know, ahead of time, what the macro will expand to.
masak that's why macros expand ASAP
I feel we are discussing at almost complete cross purposes
pdcawley masak: Yes, but you then have to run a full check on the AST they return.
def macro foo { quasi { return 42 } }; foo; should first of all expand to 'return 42' and then trigger the 'this isn't legal code' warning.
It certainly shouldn't short circuit the return from the macro, which is what you seem to be saying on issue #63. 14:13
masak `return` binds lexically to the surrounding routine.
pdcawley Not if it's quoted it doesn't.
masak which happens to be a macro.
Ven_ won't argue again over this 14:14
masak pdcawley: you're describing some other language currently.
pdcawley quoting: DO NOT EVAL THIS YET
spec for quasi as it stands is batshit insane.
if quasi isn't quoting, don't call it quasi because you will break everyone who comes to Perl 6 from every other language that has quasiquoting ever.
And you'll be dealing with them going WTF!? all the time. 14:15
And, if you do want quasi { return } to return from the macro, how am I supposed to return a quasi that will return from the _calling_ scope?
ie, where's the real 'quote'? 14:16
the one that says "do not eval this yet until it's either been plumbed into a bit of AST that will get evaulated, or is explicitly evaluated in a given environment"? 14:17
Because if that version of quote doesn't exist, there will be classes of macros that it's impossible to write.
One of the things that one might want to do within a macro is to change the semantics of 'return', after all. 14:18
Ven_ different kind of macro 14:19
let's not mix things up
masak COMPILING::return
Ven_ renaming "quasi" is the topic right now, not codewalking
masak perhaps
I agree `quasi` could be better named 14:20
pdcawley As soon as you see a macro in the stream of code, you have to treat all the parsing the parser does as tentative and liable to be thrown away. The only AST that's actually checkable in a meaningful fashion is the AST that's returned from your macro.
masak I've been thinking of simply calling it `quote { ... }`, like Julia does
pdcawley masak: special forms are namespaced?
Ven_ that's in lisp macros, pdcawley, not in perl6's
pdcawley masak: Don't do that, it's not a quote either.
Ven_ in perl6, quasis have a lexpad.
masak and then maybe renaming `{{{ }}}` to `unquote( )`
pdcawley Ven_: So I gather. I still don't quite get what Perl6 gains from this. 14:21
masak (a) it's symmetric, (b) it gets rid of the triple braces which everybody seems to love to hate
Ven_ {{{ }}} vs unquote is probably the smallest of concerns :)
pdcawley Ven_: Seems to me that you've got something that complicates implementation and limits the capabilities of the macro system at the same time.
masak I'm going to bed. will let you two arrive at the perfect synthesis in the meantime. will backlog later. 14:22
'night
Ven_ pdcawley: I'm not arguing in favor of either here
'night, masak
pdcawley I don't understand how quasi seems to be trying to be something that makes an AST but which can be returned from to a weird place. Don't like the idea of having to sprinkle 'COMPILING::' all over the shot to get control structures to work right - seems to go very much against the Huffman idea... 14:26
Ven_ pdcawley: I'd also agree quasi{} shouldn't do that, but we've established that the current quasi{} isn't a quasi
well, only `return` has such an effect, AFAIK 14:27
pdcawley Ven_: When are you _ever_ going to want quasi { return ... } to return to the place where the quasi was built?
Ven_ ask masak, I argued against this as well. I just think it's for consistency's sake with how "hygiene" works 14:28
pdcawley quasi is _not_ an execution context in the way that other blocks are.
Ven_ (I argued that hygiene should not extend to return)
pdcawley quasis having lexpads are the problem if you ask me. 14:29
Make the rule every variable used in a quasi should be either: fully qualified, declared within the quasi, or explicitly unquoted and the quasi itself doesn't need a lexpad. 14:30
By the closing '}' of the quasi, everthing in it should be resolved.
Ven_ quasi should be called "delayed-ast" or something :P
you're arguing for lisp macros 14:31
pdcawley Then when it's patched into the final AST, it acquires an appropriate lexpad and gets checked to make sure everything's in scope.
Ven_: No, I'm arguing for a useable quasi.
Can implement hygenic or unhygenic macros on top of such a quasi, it's orthogonal. 14:32
Ven_ nods
pdcawley The problem (as I see it) is the implicit unquoting of my $x = 4; quasi { $x } 14:33
Macros are hard enough without having to worry about implicit behaviour.
Ven_ it's not unquoting, again 14:35
forget about the "quasi" name
it's not quoting
so it's not unquoting
{{{ }}} is just splicing
pdcawley Then I really don't see the point of it.
Because I will bet real money that there _will_ be a need for lisp style quasiquoting, and if you can't implement that in Perl 6 there's a problem. I 14:37
I'd expect whatever perl6 provides to have a superset of lisp quasiquoting semantics, not some spavined version of it that doesn't seem to have semantics like anything else ever. 14:38
But apart from that, it's awesome :) 14:39
Ven_ well, at least we agree on that :P. 14:41
fwiw, I'm very much used to Lisp macros, so I agree with your points in a lot of ways. Whatever is we're getting, we should not be pretending it's "lisp-style"
'cept for the fact it's acting on a AST level, but.. 14:42
pdcawley Given a lispy quasiquote and codewalking, I could implement quasi. However, given quasi and code walking, I'm buggered if I can see how to implement a lispy quasiquote. 14:55
Which says to me that quasi is underpowered.
or that my imagination isn't up to the job (entirely possible)
Ven_ you'd implement perl6's quasi? means you need to walk on all the `let`, then all the quotes, and replace them there
actually, only the quotes you know will be used as an AST.
which is different in the non-homoiconic perl6 :)
pdcawley Also, if whatever we end up with isn't capable of implementing analogues with pretty much everything in Let Over Lambda, then it's underpowered.
Ven_: Where else is quasi used?
Ven_ no, the opposite
quote is used for many things in lisp. in perl 6, quasi{} is only for an AST
pdcawley Okay... perl6ish quote?
Ven_ yeah.
"late-ast"
not "quasi" :)
pdcawley To implement quasi, given lispy quasiquote, don't you just write macro quasi($block_expr) { walk the tree looking for unqualified references to variables that are only available in the COMPILING scope and haven't been declared within the quasi block and replace them with {{{ $var.perl.AST }}} }
Ven_ then you need to pass all the bindings you currently have declared to quasi{}
there's no "bindings"(ruby-like) in CL :P.
(there are pseudo packages, like MY, in 6)
pdcawley can't the macro see COMPILING:: ?
Ven_ wrong one. COMPILING is the one "down there", not the macro invoking quasi
there's OUTER::, probably 14:56
maybe.
pdcawley Ven_: But when quasi gets invoked within a macro, surely the COMPILING:: is the invoking macro's scope. 14:57
Jesus wept, but hygiene is a horrific can of worms.
Ven_ yeah, you're probably right 14:58
*g*
pdcawley Ven_: About what? COMPILING, or hygiene being a can of worms?
Ven_ COMPILING.
I like hygiene as demonstrated in scheme/racket
though let over lambda has example of a defmacro!g that helps plainly with hygiene 14:59
pdcawley Ven_: Yeah, but it's a can of worms for the implementers.
Ven_ (and defmacro!o for once-only-evaluated stuff)
pdcawley Yeah, defmacro! and friends in LoL are awesome.
Ven_ nods
pdcawley Easy to write hygienically, but all the icky power of unhygenic macros when you need it. 15:00
Ven_ I have all of that in racket and scheme as well, though :) 15:01
but yes.
a bit too ad-hoc for perl6
pdcawley ISTM that scheme/racket hygiene is relatively easy to implement when your AST is so trivially walkable and easy to pattern match against as scheme's is.
But trivially walkable and easy to pattern match against aren't things I associate with Perl. 15:02
Personally, I'd advocate for the approach of implementing an ad hoc unhygenic system first, then using that to experiment with hygenic mechanisms. Would help drive quality AST walking tools to be implemented too, which will be useful for more than just macro writers, then pick the hygenic scheme that works and enables everything that the unhygenic system did, then rip out the unhygenic stuff, write a library that implements it on top of 15:05
the hygenic scheme for those that don't want to have to rethink their libraries and are prepared to pay the performance hit.
Ven_ meh, performance 15:13
there's also the notion of templates, which are pretty different
but as always, implementable on top of macros :)
pdcawley Point being take the Perl 5 approach: implement a minimal kit of parts, use that kit to develop the Right Thing, then make the Right Thing what Perl does. 15:47
Don't decide early. 15:48
19:23 vendethiel joined
vendethiel github.com/densh/scala-offheap/blo...cate.scala macros gone wrong :P 22:12
that's what you get for being unable to generate macros from macros 22:13

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!