18:13 ilbot3 joined
moderator Discussion area for the Perl 6 toolchain | github.com/perl6/toolchain-bikeshed
nine The rest has either already been fixed and just awaits confirmation by the reporter or are straight implementation bugs. 18:13
18:27 lizmat joined
nine is the irclog already on? 18:29
flussence nope
lizmat is listening :-) 18:30
flussence moritz runs the logbot, so we'll have to wait until he's got a free moment
(AIUI, might be a while) 18:31
mst if anything important happens, I can supply
nine Just to get that thought on record: while system wide precompilation is really hard, what's easy is to provide precompiled versions of leaf modules system wide, i.e. modules that have no dependencies. 18:33
lizmat nine: this is about packaging ? 18:34
nine More or less. Packaging makes the issue much more visible. But even without packaging we currently don't use the system precomp files.
That's why I think the next step should actually be to disable precomp on installation as it's pretty useless as it is right now. 18:35
mst I think we should kill system precomp for the moment
nine However that does not mean that I want to rip out the code completely.
mst focus on making sure the ~/.perl6 version is solid
then re-introduce system precomp more thoughtfully
with a view to packaging
somebody should look at how distros do stuff like link optimisation 18:36
nine I think we should later go back and add it back in step by step maybe beginning for example with those leaf modules.
mst which is a post-install step that cares about transitive dependencies
18:36 lucasb joined
mst that might give us some insight into what we can do in the same vein 18:36
nine lucasb, alpha123: I'd need your github ids for github.com/perl6/toolchain-bikeshed 18:37
mst we must remember to drag leont in too, since while he's been doing testing, I full expect this place to end up being the intersection of installation, testing and packaging 18:38
since that's basically what's happend with perl5
lucasb oh hai, nine! thanks for the invite, but I'll probably not be editing that repo. I just came here out of curiosity :)
mst lucasb: I'm sorry, did you confuse us with people who were offering you a choice? 18:39
github id please, you can always just not use the commit bit :)
lizmat nine: for my understanding: would that mean we would need to re-install modules after each setting compile ?
because I think it's *very* important we keep at least that 18:40
mst re-install? 18:41
flussence s/install/compile/? 18:42
mst lizmat: ?
nine lizmat: no, never re-install
alpha123 nine: yeah I'm just here because it's interesting, I'm probably not that useful
lizmat nine: *phew*
nine lizmat: it's all about re-compilation
mst alpha123: I've heard that many times before. github id anyway please, even if you only ever use it once it was still 100% better than not 18:43
lizmat feels to me that the only way to fix precomp with packaging, is to allow relative paths ? 18:45
nine Btw. I hate that I didn't rip out the version directories in ~/.perl6. Because they mean that you have to re-install modules installed to your $HOME after upgrading rakudo.
alpha123 mst: github.com/alpha123
mst nine: we can turn that ibnto one command in an installer
generally, speaking as a local::lib author, I hate that I didn't put version directories in there by default 18:46
though that was mostly an XS module thing
nine lizmat: precomp can only work with leaf packages or when the precompilation happens in a post-install script. Otherwise we'd end up changing packaged files which is a no-go
mst I think post-install script may be very viable though
anything else would require packaging N*M*O*P*Q versions or something similarly insane
nine mst: true, it's totally safe to just rename the version directory to the newer version... at least right now 18:47
mst nine: yeah, and later if it ever isn't, we'll be glad to've had the choice
so ... I think solve it via tooling is correct
could even have a rakudobrew step that automatically shifts stuff for you
lizmat I can't help but thinking we would need to make precomp faster 18:48
s/faster/better/
I mean, we're talking about composability all of the time
nine Btw. if anyone has trouble following what I say, please just ask. I've been thinking about this stuff for weeks on straight. Some things may be very obvious to me and totally nonsensical to everyone else.
lizmat yet, precomp files are anything but composable
mst if precomp didn't require all transitive deps
right
nine even if you're just here out of curiosity! ^^^^
mst right, confused newbies are a *great* tool for forcing people to clarify their thinking 18:49
lizmat so before we try to fix the problem at this level
I wonder what is making it so hard to have composable precomp
nine Well our precomp is static linking with dynamic loading of the linked code. A combination that's difficult to handle.
lucasb idk much, but the only opinion I have about this is to *not* nest the site and vendor directories inside the perl/core one. I think it would be cleaner with they existed at the same level. I already told nine about this and he was kindly to acknowledge the situation. But I understand changing this now would inflict having to keep telling new users to "nuke everything".
lizmat well, to me that just feels like another layer of punisihing core devs 18:50
to an extent, I feel like I'm my boss now, many years ago
mst I don't think we have to tell new users to nuke everything 18:51
nine But having pure static linking with embedded code wouldn't be that much better. Because then installing a new version of some dependency would still not make the reverse-dependencies use the new code, confusing the users.
lizmat when I told my boss that what he wanted, wasn't possible
mst I think we'll just need to figure out how to upgrade things under them without them noticing
lizmat guess what, in the end it turned out to be possible after all
even though I was convinced it couldn't be done
nine mst: yes, we just need to detect the crewup in the install script and move some directories. Manageable.
lizmat: been there, done that :) 18:52
mst composability of precomp files is above my pay grade right now
but no matter what, I think 'fall back to .perl6, then start again carefully' is going to be a great option
means we're less likely to trap ourselves so badly
nine And it means that we have loads of time.
mst and I'd rather look at cleaning up the non-precomp CUR stuff first and making that a bit more flexible and making sure that can be packaged 18:53
and then we weave precomp into it after that
nine lucasb: please add the moving of the perl6 repo to github.com/perl6/toolchain-bikeshe.../README.md as todo
lucasb nine: can I just open a issue instead? issues are better, they can be commented on. 18:54
flussence my 2¢ is that it shouldn't put anything in "system" dirs by default, that ought to require some sort of `perl6 --I'm-doing-something-fancy` invocation
that'd neatly skip having to put a bunch of mkdir juggling right in the compunit code, because IMO that sort of stuff belongs in makefiles 18:55
mst the compunit code's mkdir juggling is hilariously wrong and awful, but I've not had a chance to go through and work out what it's supposed to be yet 18:56
flussence and since perl6 is being positioned as a compiler, there's nothing wrong with giving it a million command line switches for this sort of stuff. Every other compiler does it 18:58
nine mst: the mkdir juggling was just one of those shortcuts that made it into released code. It's really make install that should create the system repos and it does. Except for the precomp directory in each repo I think. I missed that. 19:07
But the mkdir code also helps with ~/.perl6 where make install won't do anything
mst I think "first .install() into a CUR creates the directories" is likely best
lizmat nine: is @*INC still a thing, or not ?
nine lizmat: nope, that's gone 19:08
lizmat ok, because src/core/CompUnit/RepositoryRegistry.pm seems to still contain references to it 19:09
as does ModuleLoader.nqp 19:10
nine oh in method files
lizmat I will get out the chansaw
nine CompUnit::RepositoryRegistry::files can go alltogether.
lizmat nine: will make it so 19:13
nine: so what's up with @INC in method setup-repositories() ? 19:16
nine That's just a local variable with a recognizable name 19:17
lizmat nine: ah, I see... it's only used internally
nine just for confusing the reader ;) 19:18
19:22 tadzik joined
flussence yayit'stadzik! 19:22
tadzik \\o/ 19:23
I'll lurk here :)
flussence yeah yeah, that's what I said about #moarvm... :) 19:24
tadzik :P 19:25
mst tadzik: but you're already the author of a key toolchain component accidentally 19:26
several of the things I have in mind are going to lean on rakudobrew ;) 19:27
tadzik yeah, panda also became the thing for a while :P 19:30
mst I think we'll need to pop a couple more components off the yak stack before I start having ideas that'll lean on panda 19:35
so depending on how uppity shadowcat's customers are you may yet get a fair stay of exeuction there ;)
tadzik :o 19:38
aw, shucks, I was planning to transfer all my stuff to community org and retire :D
19:39 Zoffix joined 19:40 ugexe joined 20:03 diakopter joined
diakopter such solace 20:06
[Coke] If tadzik wants, we can fork panda and keep the "official" copy in perl6 to avoid pain. 20:08
(I'm also happy to leave it where it is)
mst I trhink transferring stuff to orgs is a good thing in general 20:10
I think we can't afford to let tadzik escape just yet either way ;)
tadzik *blushes* 20:11
well, almost all I ever did in Perl 6 was dogfooding 20:12
now I don't really use Perl 6 anymore, the interests have shifted, so I don't really work on my stuff these days
(because I don't really need it)
mst right, so let's get the repos into community orgs
and you can stick around and answer stupid questions while we figure out who's running with the baton next 20:13
tadzik at times when I do use Perl 6 all my stuff just works for me, largely because of all the awesome people who quickfix it :)
lucasb tadzik++ # for all the cool stuff he did 20:16
I'm +1 for moving rakudobrew and panda to perl6 org
diakopter + 20:19
mst right, then it'll be easier for us to inflict commit bits on people 20:21
20:21 FROGGS joined
FROGGS o7 20:21
tadzik oh hai
diakopter nine: lol 20:23
FROGGS can I ask my panda/star related questions too? 20:24
nine absolutely
FROGGS cool 20:25
so, I bootstrap panda in star, and it just goes off to test and install panda
without testing and installing its deps
mst I was expecting we'd cover CompUnitRepo, Precomp, testing, installers, and packagers in here
FROGGS maybe...
mst basically the whole spectrum
FROGGS maybe I have to delete the statefile...
mst needs to summon leont
tadzik :3
flussence puts on Fake Gentoo Maintainer hat
nine FROGGS: oh yes, panda's statefile may interfere 20:26
mst amusingly, when panda bootstraps itself, it doesn't record its own deps in its statefile
FROGGS ohh, that looks better!
mst that took me a moment of confusion when trying to test 'panda --installed list'
tadzik that's a bug
nine By now I would really, really like to get rid of it :) So unless someone brings up an argument soonish, I'm gonna merge the query_repos branch 20:27
mst I was chasing lizmat's rakudobug at the time so didn't care that much
nine One thing that's keeping me from ripping the statefile out: it carries (literally) one extra bit of information: a flag if something was installed at user request or as dependency. 20:28
I don't know if panda is actually doing anything with this information, but it seems usefull.
tadzik which I put there so you can sensemakingly uninstall stuff at some point
with GC, so to say
I don't recall Perl 5 ever having that, and it always pissed me off 20:29
nine Yep, I figured.
But I don't know where else to store it.
mst lack of sensible uninstallation in perl5 is a multifacted clusterfuck 20:30
mst casts 'summon freebsd port nerd' 20:34
20:34 Kassandry joined
FROGGS btw, this is the panda patch needed for star: github.com/rakudo/star/blob/releas...ndir.patch 20:39
which is possibly not complete
and this is the rakudopatch: github.com/rakudo/star/blob/releas...ndir.patch 20:40
so, not much actually
mst flussence: were you doing an overlay for portage?
alpha123 mst: huh 20:41
flussence yep
gitlab.com/flussence/perl6-overlay 20:42
mst kassandry was pondering what'd be involved in getting a port going
alpha123 well we don't even have a perl6 port yet
or at least didnt
flussence right now I've got it down to a plain Configure.pl && make && make install pattern
FROGGS hmmm, I think I should "release" an RC0 of star... 20:43
flussence it's an overlay and not distro-upstream because moar & friends do a whole bunch of things gentoo's QA people hate
(which I've been writing down in the issue tracker there)
mst YAKS FOR THE YAK STACK 20:44
SKULLS FOR THE SKULL THRONE
diakopter oh no, he's uncorked
20:48 TimToady joined
nine FROGGS: looks sane at a first glance. I'm not sure I like it very much but can also not point to anything specific 20:57
FROGGS mhm 20:58
nine I think it's just that is pretty much a hack necessitated by the lack of design. And my utter reluctance to add API before we have a solid plan. 20:59
But clearly this is usefull and will be for distro packaging, too
FROGGS and maybe also useful for a repo that hists the modules non-locally 21:03
but the bin dir has to be in PATH, whatever you do, so it must be locally
not saying it's not a hack
mst so github.com/rakudo/star/blob/releas...ndir.patch 21:05
nine Just an observation: there's a 1:1 relation between ~/.perl6 and ~/bin and between /usr/share/perl6/site and /usr/bin
mst is basically just "pick the directory the shim scripts get generated into" ?
nine But of course, that's just on UNIX
mst: yes
mst FROGGS: of course, for non-local modules we may end up needing a smarter shim script, but yes, in theory, but oh gods let's leave that one to later 21:06
nine: though, btw, local-but-fetched-on-demand is one of the things I'd been thinking about as an alternative CUR format, with the per-repo format file thing I was talking about
(useful example to anchor the idea off if nothing else) 21:07
TimToady is this channel logged yet? 21:10
mst I believe it's scheduled to be, but isn't yet, but I have logs of everything if we discover we missed something
so "not publically, but nothing will be lost"
21:10 leont joined
FROGGS nine: if we're not going for the bindir hack, we need another way to specify the directory layout of a repo 21:11
mst I think having the shim dir configurable seems perfectly reasonable 21:12
lizmat nine: I'm looking at streamlining setup-repositories
FROGGS because currently a repo looks like: bin/, dist/, (panda/), resources/, short/, sources/, repo.lock
and no packager wants to put that in /usr
TimToady on the shared precomp thing, I always thought, if we really can guarantee immutability of a module and it's compiled results, a cron job that periodically goes off and copies identical unshared modules into a shared repo and deletes the unshared one could work out somehow and remove the objections of wasted space on a multi-user system 21:13
lizmat nine: I think I can get about 15 msecs off of each call, which which would mean something like 60 msecs improvement on loading something like Tex::CSV 21:14
TimToady *its
FROGGS so most of the repo will be in /usr/lib/.../perl6/... and the runners in /usr/bin
lizmat nine: so from 235 to 170 msecs
nine: do you think this is worth it at this point ?
(got most of the code already written btw) 21:15
mst TimToady: I suspect it may be doable via post-install scripts actually at each CUR level
oh, sorry, you're talking about space usage
tell them to get a better filesystem already ;) 21:16
(seriously, the world is moving towards single copy fses anyway)
leont always imagined packagers creating their own CUR subclass, to make p6 do whatever they wanted to
mst that wouldn't be terrible but I'd like "what they want it to" to be something I approve of, so making sure it leads them down that path would be preferable ;) 21:17
21:18 masak joined
FROGGS leont: yes, but we've got open gh issues about packagers that want to continue packaging star 21:25
leont: they'll learn how to create a nice cur eventually
21:26 thzre joined
mst also, frankly, vendors all doing things subtly and incompatible differently has been the bane of our fucking lives on the p5 side 21:26
I would rather minimise the extent to which they have to do that
and instead give them sufficient knobs to twiddle that they can have it "their way" atop a standard set of features
FROGGS most packagers (at least the few I know) are not hackers that want to ship their stuff 21:27
they use want us to fiddle it right and provide the right knobs 21:28
leont Fair enough 21:29
FROGGS but cur surely leaves options, and I like that
masak hi everybody. I'm just a Perl 6 user, but I'm very interested in #toolchain stuff, to the extent it can impede my productivity with Perl 6 ;) 21:30
FROGGS and even if a cur exists that goes in the wrong direction, they can fix it, because nothing relies on what the cur actually does internally 21:31
... or filesystem wise
masak: :o)
mst right, time to go meet $*GF
later all
FROGGS o/
FROGGS .oO( his dynamic girlfriend just callframed... ) 21:32
hoelzro o/ masak
nine lizmat: sure :) 21:42
lizmat okidoki :-) almost done
FROGGS ***ALL*** - just made an RC0 of rakudo star, as of today's codebase: froggs.de/perl6/rakudo-star-2016.01-RC0.tar.gz 21:43
***ALL*** - please test it and report bugs to the rakudo/star repository, but keep in mind it is an RC0, so dont update your production servers just now
(just because it is more visible here)
hoelzro \\o/ 21:44
FROGGS++
I like that ***ALL*** convention
FROGGS :o)
and it is not a netsplit, even :o)
nine TimToady: I'm not sure anyone has objections because of wated space (precomp files aren't that large) but because of every user on a system having to wait for the first compilation 21:45
FROGGS hmmm, does this problem exist? 21:46
not that many users touch my laptop or my servers actually
nine It's the one concrete issue that yesterday's discussion brought up IIRC, so it seems to be for at least some people. 21:48
hoelzro nine, TimToady: I'm pretty new to thinking about the whole precomp thing, but if a user is installing a module without admin perms, it seems like it should belong to that user only
admins should be able to create "global" installs of precomp'd module
nine hoelzro: that's what currently happens 21:49
hoelzro much like site lib and local::libs
nine: the current issue is that there are paths embedded in the precomp files, right?
nine if the site lib is not writable by the user, panda installs into ~/.perl6
FROGGS and the precomped product will always be a different one, because of timestamps etc
hoelzro true
nine hoelzro: that's one of the issues that make packaging harder
hoelzro right
I' 21:50
ugh
I'm not sure how I feel about ~/.perl6 by default, but that's a whole other story
nine hoelzro: add it to github.com/perl6/toolchain-bikeshed :) 21:51
hoelzro is that a panda policy or a CUR policy?
will do=)
nine it's panda's policy
hoelzro alright 21:52
which scripting languages (other than Perl 6 and Python) store compilation artifacts on disk?
nine leont: I've fantasized about a rpm database based CUR, but I'm not sure if it makes sense
TimToady well, if you use C as a scripting language... :P 21:53
hoelzro TimToady: ;)
leont I've tought the same (well, about deb though)
hoelzro I should clarify: languages that compile to a disk artifact without explicitly being asked
nine rpm does have name, version, vendor and more so at least that part would match
hoelzro import my_module in Python generates my_module.pyc transparently
another system that's quite different that may be useful to think about is .NET's GAC model 21:54
I can't speak to how well it works in practice, but it may provide good examples of what not to do =P
nine FROGGS: everything about those scripts is a hack. That repos have a bin/ directory was a quick hack to get panda going. Now we get $bin-dir. 22:01
FROGGS nine: no, these scripts are fine, really 22:07
nine: or do you think that dists are about modules only?
nine No, don't mind what I'm saying too much. I should be in bed instead of uttering nonsense :) 22:08
FROGGS then let's continue on another day :o) 22:11
gnight 22:12
lizmat gnight FROGGS 22:14
hoelzro night FROGGS 22:20
diakopter
.oO( .NET has a precomp thingie (also system-wide and per-user) called ngen... )
 22:37
(native gen)
flussence ISTR niecza's readme advised running that for something or other 22:38
diakopter but it's opt-in because it strictly disables most of the JIT... at least, 8 years ago....... 22:39

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!