Discussion area for the Perl 6 toolchain | github.com/perl6/toolchain-bikeshed
Set by moderator on 8 January 2016.
00:27 leont joined 01:50 japhb joined
ugexe ah, pwned by EVAL 'use 007' ($dist<name>) instead of EVAL 'use _007::Runtime' ($dist<provides>.keys[0]) 02:28
06:35 Zoffix joined 09:10 llfourn joined 09:15 hankache joined 10:14 FROGGS_ joined
masak oh -- is there something I can do to help on the 007 side, or is it strictly a zef thing? 10:33
nine panda now actually checks a dist's Perl 6 version requirement and errors out if it cannot be met. This should make it much easier to diagnore installation issues caused by the user having a too old Rakudo. 11:39
12:24 leont joined
nine panda will now fall back to curl and then to wget if downloading meta data failed. This should fix the proxy issues. 12:27
I wonder why we use git:// URIs in the ecosystem. panda seems to be dealing with https:// just fine. Although I can't quite figure out why... 13:02
The second "why" is because panda's project-from-git really checks for ^git:// 13:03
leont Why wouldn't we? 13:16
nine You mean why wouldn't we use git://? Because github recommends https:// as more reliable. Especially in restricted network environment. 13:34
JimmyZ +1 to https 13:35
leont Restricted network environments is a good reason 14:01
(though I still hate it for development) 14:02
14:26 hankache joined
llfourn nine: I have PR to docs which uses precomp to replace EVAL slurp. It may be of interest to you github.com/perl6/doc/pull/334/files :) 15:38
16:17 autarch joined
nine llfourn: I _like_ that! 16:26
llfourn nine: thanks! I thought you might.
hoelzro nine: not to mention that git:// is subject to MITM attacks
leont: ā†‘ 16:27
leont Is it?
hoelzro mhmm: git.661346.n2.nabble.com/git-protoc...01259.html 16:28
leont I suppose it is when the host hasn't seen the server yet, but given almost all repositories use github, that seems rather difficult to pull off 16:29
mst also, if you actually care about integrity you should be getting checksums via a separate route
hoelzro leont: true, it's more of a theoretical than practical attack
mst: *nod* 16:30
it just occurred to me that this channel is probably slow enough to effectively backlog \\o/
I was going to ask a question about moving away from GH as the canonical place to fetch modules from 16:31
Zoffix GH is just temporary. jdv79 et. al. are working on the P6 PAUSE/CPAN/MetaCPAN stack 16:32
mst uhhh
hang on
we have jdv79/ranguard's prototype of putting stuff onto cpan via pause
Zoffix It works in fact. You can upload Perl 6 modules to pause.perl.org *right now* 16:33
mst yes, and cause carnage and problems because nobody knew anybody was going to do that unbtil ranguard did it
so everything downstream is completely unprepared
autarch I really hope that whatever the end solution is that I never go to metacpan.org and see Perl 6 stuff
Zoffix No, it'll be a separate site 16:34
hoelzro *whew*
mst there will be an mc6
but there's lots of things that trail the PAUSE aupload feed
and basically every time somebody uploads a Perl6 dist half of them crash
and then distro packaging teams start asking the perl5 toolchain "wtf are these? should we be packaging them? if so how?"
and since nobody talked to the perl5 toolchain team first our answers have mostly been "they did WHAT?! what the flying fornication?!" 16:35
so I would prefer that we don't encourage -lots- of people to start uploading until we're a little more certain of what we're doing and have some public messaging explaining what and why
leont Yeah, it doesn't seem very useful 16:36
moderator Fire is step THREE! | github.com/perl6/toolchain-bikeshed 16:36
mst it may be that slotting them all together will be 16:37
and it may be that cpan is the best distrribution mechanism to use for tarballs
autarch I would note that the PAUSE website for authors is not that great to use, and I'd be very happy to not have to use it for p6 stuff
mst but I think everything else is "this is an open question because it's clear nobody's actually thought it through yet, or if anybody did, they're a distinct set from the people currently trying to do things"
leont would like to see a solution based on dumb data storage, but smart data indexing 16:38
Zoffix autarch, yeah, and the argument there is there are only so few hands to code stuff, so doing-something-new() is starved for resources
autarch yeah, I get that
leont Having non-turing-complete versions (end other requirements?) would help there, since then we could ask the index "I need a Foo that satisfies these requirements" and it can actually be helpful 16:39
nine mst: FWIW I actually tried bringing the Perl 6 on CPAN topic up at 2014's Austrian Perl Workshop, at the 2015 QA Hackathon and at YAPC::EU, but Perl 5 people didn't seem interested.
autarch nine: the problem is that the people who maintain the Perl 5 CPAN stuff are all already busy, and presumably not that enthused about doing more work for something they may not ever use 16:40
nine With few exceptions like Andreas with whom I had a very interesting chat about topics like this :)
mst I think, understandably, nobody was really convinced at that point perl6 was ever going to ship
nine yep, that was the vibe I got 16:41
I guess this will (slowly) change now :) 16:42
mst and andreas, sadly, barely talks to the rest of the toolchain people 16:43
autarch well, don't expect every p5 person to care about p6 - some will, some won't - there are plenty of p5 people who really don't like p6
(I mean don't like the language)
mst personally, ideally, I think we would do well (ab)using the 'giant distributed tarball store with user accounts' part of cpan 16:44
nine autarch: I don't expect everyone to care. I just think, it will be harder to deny that Perl 6 exists :) Though if Perl 6 matters is a different question.
mst and then trying to basically avoid the rest of it
but that may require more tuits than are available
I dunno
I'd just like to not run before we can walk
nine Yes, I'd love to be able to use CPAN just as a tarball store as a first step and be able to download those tarballs with panda 16:45
Or even just use the tarballs for distro packages
leont You might want to write an Archive::Tar and a gzip/bzip2/xz module then, those are rather missing right now 16:46
nine leont: I'm me. I'd not hesitate to use Inline::Perl5 for that ;) 16:47
leont This is toolchain, you can't assume all platforms will have p5 16:48
nine panda already does
and rakudo's build
Well panda assumes you have a "prove"
leont I know panda does, and have a plan to change it
mst though I can imagine win32 could easily have neither perl5 nor tar
leont Exactly 16:49
mst honestly though I think "needs perl5" is not actually that high on the list of problems
autarch yeah, p6 really needs to get to the point of being self-bootstrapping if we expect more adoption
leont panda's architecture doesn't lend itself for a drop in replacement for TAP::Harness, but it's not like the module isn't written or working
autarch I should say the toolchain does - having the actual build require p5 isn't as big a deal, I suspect
leont Requiring it on build is suboptimal, but not something urgent to fix 16:50
Requiring it at runtime is rather embarrassing
autarch yeah
I do note that you can start Archive::Tar etc by just looking for CLI utilities - as an intermediate measure 16:51
leont AT isn't that complicated, but there are lots of gritty details to it as far as I understand (because of 40 years of history).
A straight port of p5's AT may not be a bad starting point, because it has dealt with various of those issues 16:52
ugexe zef can download/untar from metacpan 16:54
mst I wonder if the PAUSE issues would be solved simply by removing the PErl6/ dir from the upload feed and minting a separate upload feed 16:55
that might stop people seeing things they're confused by
ugexe or it could a few days ago. need to update a few things to use DependencySpecification again
mst ok, this may be a stupid question but - how did we end up with panda and zef? just two people working on ideas independently? or is there some philosophical difference that caused a deliberate split? 16:56
(tried as hard as I could to make the question neutral, I don't have an opinion here, I just don't know the history) 16:57
nine There's also ufo
leont Does anyone actually use ufo? 16:58
ugexe panda would fail to work for days at a time due to a dependency or something else. so zef was meant to be more plugin based and without any (perl6) external dependencies so it was easier for an end user to keep working
also to do things different from panda for the sake of doing it different. i.e. panda used CU::L::File and zef used CU::L::Installation 16:59
mst at this early stage, doing things differently for the sake of it honestly sounds like a great idea (previous sentence was not sarcasm) 17:00
leont I suspect zef would be an easier target for integrating TAP::Harness than Panda does (maybe also because it doesn't do much right now in that corner) 17:03
autarch m: say False ~~ False 17:04
camelia rakudo-moar 725348: OUTPUTĀ«Falseā¤Ā»
autarch is that really intentional?
doh, wrong channel 17:05
ugexe it may well be. i have not yet gotten around to using your test harness to see but its been my intention for it to be the default other than running `perl6 -Ilib t/file.t` on each file 17:07
nine ugexe: I think zef doesn't handle resources correctly yet. 17:10
Other than that I like what I've seen so far. 17:11
leont It's fully functional except the async parts (rakudo bugs) and currently the ANSI coloring (a rakudo regression)
ugexe yeah there is still more work to do on it like parallization, a way for the phases to talk to each other during parallelization, terminal output prettification (instead of the debug messages), although those 3 were implemented in the previous version 17:14
17:14 ranguard joined
ugexe leont: what parts of it make it difficult to use with panda? it may make it easier for me to avoid them 17:15
nine ugexe: how does zef know that a dependency is already installed? 17:16
ugexe ive seen your .resolve fork so i havnt put much effort into that. it evals the dist's identity (Distribution.Str()) and if that fails its evals the first provide's spec string 17:17
previously it would read the manifest 17:18
a manifest that isn't multiple levels deep like the previous one just for mapping some basics like where to find the installed dist meta might not suffer from the slowness of the previous manifest 17:21
mst nine: oh, when you get a minute, can you try and summarise what you got from your conversations with andreas? I doubt they're written down anywhere 17:32
nine mst: I'm not sure I can remember all that much. Writing it down would have been an excellent idea. I know it was about Perl 6 on CPAN, cross language dependencies and how rpm already solves many of the problems we struggle with. 17:41
ugexe: If you have any input on .resolve, now would be the time :)
mst rpm is unusual in that it can handle having multiple versions of the same package installed so long as they don't have conflicting files
it continues to sadden me that /deb doesn't
I put this in at least one YAPC talk
ugexe nine: im not sure really. the method of a manifest that only maps dist identity + provides identities to the actual dist file allows a way to only read the appropriate meta files without having to recursively read them all until you find what you need. the drawback being its more fragile, like when someone manually deletes a distribution. im guessing either way its specific to each CU::R 17:44
i think part of the problem of the previous manifest is it had to read like 6 levels deep of every single distribution to find what it needed 17:45
mst I'm pretty sure I can create a much better repo internal format 17:46
but I think step zero is to make repo formats versioned+pluggable
nine ugexe: I think the biggest issue with the previous manifest was that it grew with each installed distribution and rakudo's JSON parser is dog slow 18:15
I guess once we provide proper uninstall support, people will be less prone to poke at repository's private files manually. And uninstall is mostly a decision on language versioning implementation away 18:17
Right now everything is blocked on that
mst maybe we should port JSON::Tiny
ugexe but each distribution wouldnt have to create a huge structure like it did previous, it only maps to where the huge structure is located for each dist. i dont know if that works or not
there was a new on hackernews a few months back about "the fastest json parser" or some such, where it would skip parsing over levels of the data structure that weren't needed. so if you wanted a key that started with 'F' and it reached a key that started with 'A' it somehow parses only whats needed to find the next key. a specialized manifest parser (or a format that lends itself to it) could 18:18
sidestep slowdown when 100s of modules are installed
nine There's another reason why we would want to avoid a large json file like that: it sucks for packaging modules. With the current structure, installation can really mean just dropping some files and fireing off precompilation, except for the short name lookup files. But those could be replaced by directories. 18:21
mst I think each dist should have some sort of JSON meta file, plus a tree of installed files 18:22
then whatever extra indices and dependents like short name and precomp can be a derived product of that
things you have to regenerate are best kept pure functional 18:23
nine I think I'm gonna implement uninstall even before we merge .resolve. Just to make sure the API fits nicely together
mst if you see where I'm handwaving
ugexe i cant speak for packaing modules, but size wise 1000 modules with a key-pair mapping module name to the actual meta file would be smaller than the previous manifest with like 50 modules installed
mst right, a single-file global json manifest is not something we want to be loading at runtime, ever
nine With a little adjustment to the short name lookup files, we would only have to parse the JSON data of the exact dist that's going to be loaded
mst a directory of file-per-dist json should provide the same features and be lots less painful
ugexe a manifest manifest 18:24
mst nine: right, which is why I'm thinking about repo format versions
maybe I should write down a textual defintioin of what ::Installation currently has on-disk-format-wise
and then we should ask, separate from the code, what's wrong with that
nine I think someone created a github repo for documents like that... 18:25
mst :D
nine Add version, auth and api to the short/ files and you can pick the matching dist right there. Of course this begs the question of what exactly is allowed in each of those fields. Would we allow white space? Newlines? 18:28
mst since the short/ files aren't meant to be consumed by humans 18:29
we could use null separated values
nine Oooh...true 18:31
You know when you've done Perl too long when you try to think about problematic characters and the worst you come up with is \\n
Why is it always that literally when writing the third line of a supposedly straight forward job, you find out what's actually quite hard about it? 18:43
mst hey, this is a toolchain channel. finding out before you've finished and deployed it is always a win. 18:44
nine Of course, I could make my life much simpler by lowering requirements. If the user wants to uninstall a module that's depended on by others, who am I to deny it? 18:45
Because finding out the answer to "is this thing still needed" is surprisingly difficult 18:46
mst write a low level one that just uninstalls
we can add a more intelligent one later
intentionally stupid often works well at this level - see things like cpanm, fatpacker, etc.
nine Also we'd want to split up these two tasks into different methods anyway. 18:54
mst right, so at least you can shore off the crazy shit to the outer method :)
also, consider that you can't fully find 'depended on' anyway 18:55
since e.g. there could be a CUR that stacks on top of this one that has something that depends on it
(I've seen 'breaking one local::lib stack updating things for another' happen, sadly) 18:56
maybe just provide a things_depend_on_this() method for the CUR and have uninstall document it exists and people can call it if they want and then die/warn as they prefer 18:59
mst sees our job wrt CUR as primarily to provide mechanism and let external tools use that to implement policy, roughly
& 19:00
19:00 patrickz joined
lizmat " <mst>\tand since nobody talked to the perl5 toolchain team first our answers have mostly been "they did WHAT?! what the flying fornication?!"" 19:21
sorry mst, but that is *not* true
we already had a meeting about this at the QA hackathon in Lancaster
and we talked about it at the QA Hackathon in Berlin
sorry, I don't think P6 devs can be held responsible for P5 devs not taking P6 seriously 19:22
it was and is a deeply felt sentiment by Andreas Koenig that there should be *ONE* CPAN
and personally, I'm not ready to abandon that 19:23
.tell mst please check my monologue at irclog.perlgeek.de/perl6-toolchain/...i_11859804 19:24
TimToady that's a fine sentiment, but we also designed Perl 6 to not care where its modules come from 19:26
autarch I'm not sure the benefits of one CPAN is really worth the risk
it's nice to reuse architecture and all that but any breaking critical p5 stuff that currently works seems like a real risk
and while I'd like to see p6 succeed, I need p5 CPAN to keep working for my $day-job in the forseeable future 19:27
Zoffix lizmat, the .tell bot isn't here
TimToady
.oO("Do not put your new wine into old wineskins, or the wineskins might burst, and the wine will be spilled out and ruined.")
we can use CPAN if it won't burst :)
lizmat TimToady: I wrote part of the design where modules could live just about anywhere, so I'm aware of that 19:28
moderator Fire is step THREE! | github.com/perl6/toolchain-bikeshed | Channel logs: irclog.perlgeek.de/perl6-toolchain/today 19:28
lizmat that being said, I do think that at least PAUSE should be able to accept P6 distributions 19:29
and afaik, it already can
*without* interfering with PAUSE itself
Zoffix right
lizmat now, that it may break stuff out there: sure, this can and will happen 19:30
but afaik, the fix is easy: ignore anything that is stored in a Perl6 subdirectory from the top of a home dir of a PAUSE user
it's even easy enough to be part of an rsync filter 19:31
so, if anything, this is an example of bad communication between PAUSE / CPAN and its P5 user base
*not* of P6 devs just blatantly trying to destroy the P5 CPAN infrastructure 19:32
sorry, but I'm quite livid at mst at the moment
lizmat goes away fro a bit to cool off
19:51 llfourn joined 19:59 patrickz joined 20:15 hankache joined 20:16 patrickz joined 20:52 llfourn joined 21:23 mohij joined 21:33 llfourn joined 23:05 patrickz joined 23:11 mohij joined 23:16 patrickz joined 23:30 mohij joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!