Perl 6 language and compiler development 2.0 | Logs at irclog.perlgeek.de/perl6-dev/today | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
Set by moderator on 2 May 2018.
00:16 quotable6 joined, nativecallable6 joined, coverable6 joined, evalable6 joined, bloatable6 joined 01:03 Kaypie joined 01:59 ilbot3 joined
moderator Perl 6 language and compiler development 2.0 | Logs at irclog.perlgeek.de/perl6-dev/today | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
02:32 cog_ joined 02:53 softmoth joined 03:55 softmoth joined 04:23 unicodable6 joined 04:39 softmoth_ joined 05:16 ufobat joined 05:53 cog_ joined 06:03 AlexDaniel joined 06:06 softmoth_ joined 06:41 softmoth_ joined
[Tux] Rakudo version 2018.04.1-37-g1447f41e6 - MoarVM version 2018.04.1-51-g6d69c6b08
csv-ip5xs0.864 - 0.890
csv-ip5xs-208.576 - 9.124
csv-parser37.388 - 38.736
csv-test-xs-200.464 - 0.477
test9.298 - 9.635
test-t2.460 - 2.556
test-t --race1.084 - 1.145
test-t-2043.344 - 44.679
test-t-20 --race16.290 - 16.981
 06:44
07:10 robertle joined 07:16 ZofBot joined, huggable joined, buggable joined 07:25 AlexDaniel joined 07:27 [TuxCM] joined
nine Another interesting piece: NQP's own modules (in install/share/nqp/lib) do not differ between before and after. It's only rakudo's blib folder 07:31
Which is quite logical, since NQP just didn't get rebuilt in the test :) 07:39
And indeed, I see differences in the files when I rebuild my NQP! Finally something to work with. 07:41
07:48 softmoth_ joined
lizmat Files=1239, Tests=76313, 322 wallclock secs (15.88 usr 5.67 sys + 2216.91 cusr 222.70 csys = 2461.16 CPU) 08:10
AlexDaniel squashable6: next 08:25
squashable6 AlexDaniel, ⚠🍕 Next SQUASHathon in 1 day and ≈1 hour (2018-05-05 UTC-12⌁UTC+14). See github.com/rakudo/rakudo/wiki/Mont...Squash-Day
09:08 Ven`` joined 09:09 Ven`` joined 09:12 cog_ joined 09:26 [TuxCM] joined 09:29 stmuk_ joined 09:50 stmuk joined
Geth nqp: 026f620fcc | (Elizabeth Mattijsen)++ | tools/build/MOAR_REVISION
Bump NQP to get the latest Moar goodies
 10:35
¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...2-gdf91ba7
lizmat .tell nine if you now on HEAD and change NQP_REVISION to 2018.04.1-21-g026f620 10:40
yoleaux lizmat: I'll pass your message to nine.
lizmat .tell nine then you will see a failing install with a different error each time 10:42
yoleaux lizmat: I'll pass your message to nine.
lizmat .tell nine if you do a configure: if you just do the "make install" again, the error stays the same
yoleaux lizmat: I'll pass your message to nine.
lizmat nuking .precomp dirs fixes the issue 10:45
lizmat refrains from bumping NQP atm
afk for a bit& 10:55
samcv so looked into implementing :ignoremark for Str.split. it's simpler for regex because we just return the match. so 'á' ~~ /:ignoremare a/ returns á as the match 11:01
but let's say we have a string 'yAbcÁdef' and split on A 11:02
do we get: 'y', 'bc', '´def' or do we get 'y', 'bc', 'def' 11:03
where the 1st one we split literally on A and so when we see Á we split it and put the ´ where it would go
or we just dump it. and if we dump it, then if we use :v, which returns the needle matches too, then we'd have 'y', 'A', 'bc', 'Á', 'def' 11:04
though [Tux]'s issue we're trying to solve is that splitting on quotation marks `"` doesn't work when we have "´ such that the ´ attaches onto the " and so it doesn't split. and in that case we don't really want to be splitting Á which is a single codepoint into A + ´ which could alter the result 11:05
11:05 pmurias joined 11:06 brrt joined
pmurias samcv: what would be the use of the later case of having the dangling combining characters be in the split parts 11:06
?
samcv no clue
but i agree that seems mostly useless. which basically makes me believe the former case with ignoremark is what we'd want 11:07
pmurias 'y', 'bc', 'def' seem to be what we want
* seems
samcv pmurias: yeah
but that doesn't result in a correct fix for [Tux] since while it may work with " which is what [Tux] needs, but if we split on 'a' it would alter the output 11:08
and [Tux] is *really* wanting something like i forgot the flag name, but it's in the design docs
but it does a codepoint based overview (for doing regex)
11:09 ExtraCrispy joined
|Tux| pmurias, utf8-c8 11:09
samcv which we don't have implemented yet since it's not that useful. since you don't really do regex at codeponit level
[Tux]: though it would apply for non utf8-c8 if you just care about the codepoints only
|Tux| true
samcv so i would want to implement a new nqp::index_foo which indexes but returns the grapheme number as well as the codepoint within the grapheme 11:10
|Tux| but the point here is that I might have binary data (from jpg/gif/gz/xz/bzip2/....) that is encoded as utf8-c8, in which the *semantics* of what are valid combinations in Unicode do not really matter
it is theoretically possible that a binary field translates to only combining codepoints, which makes absolutely no sense in a real utf-8 world 11:12
AlexDaniel samcv: by the way, how is timotimo doing it in JSON::Fast? 11:14
I recall there was a similar problem
github.com/timo/json_fast/blob/mas...st.pm#L197 11:15
uhhhh
also, I don't see how 'y', 'bc', 'def' is what we want 11:24
or, at least, not in casse when you'd actually need such a feature
in case of json and other stuff, you may sometimes want to store a single combiner as a string
11:25 El_Che left
AlexDaniel so that'd be "`" 11:25
if you throw away the combiner, then you're throwing away the useful data
case*
robertle I am trying to sum up the options for debian around module packaging. the page is only half done, especially everything from "Possible Solutions" onwards, but it would be fab if some of you could review it anyway to check whetehr what I am saying about rakudo is correct. but also whether you think there are some possible/likely future changes that I should be aware off: 11:27
wiki.debian.org/Perl6PreCompProposal
AlexDaniel weekly: wiki.debian.org/Perl6PreCompProposal
notable6 AlexDaniel, Noted!
AlexDaniel Attempting 'MyShit' as a pragma xD 11:29
robertle this is very much work in progress and under discussion. let's not put it in the weekly yet...
we should really have a MyShit pragmma in core... 11:30
it's disappoiting that I have to define such a central language feature myself
AlexDaniel robertle: well, the point of weeklies is to report progress :) 11:32
or so I thought? 11:33
robertle yeah, just not sure it is progress yet :) 11:34
pmurias AlexDaniel: throwing away the combiner isn't the point of ignoremark that? 11:35
AlexDaniel robertle: the con for “Re-Pre-Compile at module install and rakudo upgrade” is that it will take forever to upgrade rakudo if you have a bunch of modules installed
nine ~
yoleaux 10:40Z <lizmat> nine: if you now on HEAD and change NQP_REVISION to 2018.04.1-21-g026f620
10:42Z <lizmat> nine: then you will see a failing install with a different error each time
10:42Z <lizmat> nine: if you do a configure: if you just do the "make install" again, the error stays the same
AlexDaniel pmurias: sure, but this discussion was started in response to a practical problem that modules have 11:36
pmurias: we can of course implement the way you say, just because it makes sense or whatever, but so far nobody asked for it I think?
nine robertle: "The 5535...B91B is a hash of the module path to load" is just wrong 11:37
robertle: "The most basic problem with this is that we need to clean up after packages, but there is no way we can tell which precomp files belong to which package". Why don't you know that? Isn't the rob of the packaging system to keep track of...packages and what is installed? 11:38
pmurias AlexDaniel: implementing stuff in a way that doesn't make sense just because it's useful doesn't seem a super good way to design the language 11:43
nine robertle: in my opinion, there is simply no real alternative to "Ship pre-compiled files in packages". Unless you want to go full Gentoo. The whole point of using a binary distribution is to compile once on a build server instead of millions of times on user's machines. Getting updates for module packages on rakudo upgrades should not be an issue. The same has always been true for Perl 5. Perl packages
are rather tiny anyway.
robertle: It's also not just Perl. I get updates to literally a thousand of texlive packages every so often. That probably costs about 2 minutes. 11:44
AlexDaniel pmurias: sure, great, let's implement feature that nobody needs 11:45
robertle right, before we get to the pros/cons, which I haven't really worked on much yet, let's look at the facts first!
nine: "The 5535...B91B is a hash of the module path to load" if that is wrong, what is it a hash of then? 11:46
ilmari nine: python packages on debian byte-compile at install time
as do elisp packages 11:47
robertle regarding "The most basic problem with this is.." that is of course only a problem if we do *not* ship the precompiled files, will clarify
ilmari but there the byte compiled files go into predictable locations
/usr/lib/python3/dist-packages/$module/__pycache__/*.elc and /usr/share/emacs24/site-lisp/$package/*.elc, respectively 11:48
eh, *.pyc for the python stuff, not .elc 11:49
robertle also note that the *.pyc files are not very tighly coupled to the python version, so they can stay the way they are pretty much al the time
debian compiles them on install, but does not re-compile them on python upgrades
pmurias AlexDaniel: I'm not suggesting that either, but I'm super opposed to subverting stuff in a silly/weird ways to workaround for the lack of way to process strings in code-point focused manner 11:52
AlexDaniel it's not silly and not weird. You can call it :leavemarks or whatever if the name is the problem for you 11:53
greppable6: ignoremark 11:54
greppable6 AlexDaniel, 12 lines, 3 modules: gist.github.com/5687020e7adf8e95f3...8eeb47e830
nine robertle: that hash is an opaque value uniquely identifying a precompiled file that is useable with the current compiler and repository state. 11:55
pmurias AlexDaniel: the underlying problem is that users want to split on a codepoint sequence? 11:56
robertle nine: really? aren't there multiple parts to it? I thought the first part is teh compiler id, and the last aprt is refering to the module? Although I think it's not the "path" of the module, but more like an module spec ... 11:57
nine Why is Debian wasting their user's resources for compilation? Doesn't make sense to me :)
AlexDaniel pmurias: arguably, yes 11:58
robertle we are not at that point yet, just need to get the facts straight first, then come up with a few proposals and see which one makes most sense...
nine robertle: the point is: it's an implementation detail that is subject to change. Your question is like asking what the numbers in /var/lib/pgsql/data/base mean.
robertle that is certainly a good point, noted 11:59
but regarding that hash: my reading of the code is that when you "need" a module, a dependencySpecification is created. with a short name, version, author etc 12:00
nine Precomp directories are nothing like __pycache__ directories. They are opaque databases.
robertle we then create a precomp id from that shrot name via CompUnit::PrecompilationId.new-from-string 12:01
did I read that wrong?
AlexDaniel .tell El_Che prerelease is up and tested :) That's pretty cool. I'd try to do these regularly, and maybe I'll even automate it 12:03
yoleaux AlexDaniel: I'll pass your message to El_Che.
nine robertle: it's a combination of the dist's long name and the module name 12:05
That's the current implementation at least. There have been others.
pmurias AlexDaniel: one potentially crazy solution would be to expand the utf8-c mechanism to decode something as a string as a sequence of codepoints 12:06
robertle nine: what do you mean with "dist's long name"? I don't quite understand... 12:07
nine robertle: the "name", "ver", "auth" and "api" fields from the distribution's META6.json 12:08
robertle right, thanks! 12:09
regarding the pros/cons: the big con of shipping precompiled files is that we end up with a very tight coupling between module packages and the runtime. You can only upgrade them together (and we can only migrate them from e.g. unstable->testing together) 12:10
I am curious to hear from the python folks, because they do not seem to have this problem, yet elected to precompile on installation 12:11
AlexDaniel .tell timotimo some discussion regarding .split and " marks with combiners (somewhat relevant to JSON::Fast): irclog.perlgeek.de/perl6-dev/2018-...i_16124461 12:12
yoleaux AlexDaniel: I'll pass your message to timotimo.
AlexDaniel pmurias: that might work… 12:13
and it'll also “fix” other operations, not just split
nine robertle: the same has always been true for Perl 5
robertle nine: really? very interesting! but why? a typical perl5 module package only contains sources, and when we upgrade the perl5 runtime it will also compile and run the older source just fine... 12:16
in most cases that is ;)
nine robertle: there are loads of XS using modules and those are compiled against the perl interpreter. That's why they are installed in a versioned path like /usr/lib/perl5/vendor_perl/5.18.2/ 12:17
robertle right, I'll look into the XS case 12:18
very interesting case indeed, thanks! 12:19
12:32 cog_ joined 12:38 brrt joined 12:47 undersightable6 joined 13:01 bisectable6 joined 13:10 bisectable6 joined 13:13 committable6 joined 13:15 committable6 joined 13:22 committable6 joined 13:26 undersightable6 joined 13:31 undersightable6 joined
AlexDaniel Zoffix: *.party websites seem to be down gist.github.com/55cc92beb7f1aa4681...581f9a61e9 13:35
timotimo AlexDaniel: didn't he drop .party because .party is ridiculously expensive after the first year? 13:41
yoleaux 12:12Z <AlexDaniel> timotimo: some discussion regarding .split and " marks with combiners (somewhat relevant to JSON::Fast): irclog.perlgeek.de/perl6-dev/2018-...i_16124461
13:44 skids joined
AlexDaniel 1.89$? 13:51
timotimo interesting, apparently it's now the same for every year? 13:52
AlexDaniel I don't know, I use www.namesilo.com/ and they always charge the same for renewals 13:53
well, unless it was some super promotion, but still 13:54
timotimo it seems like every registrar can do whatever they want
1&1 for example charges 7.17 for registration and 30 for renewal
AlexDaniel O_O
timotimo godaddy has 20 for registration, 40 for renewal
AlexDaniel that's for… .party?
timotimo yup 13:55
AlexDaniel just transfer it to namesilo then?
:)
timotimo godaddy apparently doesn't show the renewal price anywhere on their ordering process 13:56
lizmat afk again& 13:57
timotimo m: sub can-turn-into(Str $string, Any:U $type) { return so $string.$type; }; say can-turn-into("3".Int);
camelia Too few positionals passed; expected 2 arguments but got 1
in sub can-turn-into at <tmp> line 1
in block <unit> at <tmp> line 1
timotimo m: sub can-turn-into(Str $string, Any:U $type) { return so $string.$type; }; say can-turn-into("3", Int); 13:58
camelia True
timotimo that's fun
nine m: sub can-turn-into(Str $string, Any:U $type) { return so $string.$type; }; say can-turn-into("0", Int); 14:03
camelia False
nine I broked it!
Geth nqp/pluggable-spesh: 9a82ca26c2 | (Jonathan Worthington)++ | t/moar/52-pluggable-spesh.t
Tweak spesh plugin test to actually hit OSR

Also can reduce the counts a little.		 15:21
15:53 cog_ joined
b2gills I wonder if its worth noting on wiki.debian.org/Perl6PreCompProposal that how .precomp works can be altered if someone comes up with a good idea how it should change 15:57
15:59 robertle joined
samcv regarding doing .split but only looking at codepoints, i found in the design docs a :codes modifier for regex. design.perl6.org/S05.html#Modifiers 16:14
though it also names: graphs, chars, and bytes. and i'm not sure how graphs and chars are supposed to differ… 16:15
but split(['foo', 'bar'], :codes) seems sane to me
timotimo so, what do you think, should i rewrite JSON::Fast to work directly with Uni objects? (or bufs of 32bit integers or whatever) 16:17
b2gills samcv: Note that S** are old, and in most cases predate any usable implementation. So take anything you read there with a grain of salt. 16:18
samcv b2gills: yep for sure. heh. 16:19
jnthn At some point we'll have to actually do a bit more of the design work for Uni...
samcv "UTF8-c8 is all anybody will ever need" — Bill Gates, probably 16:20
jnthn Nah, that's solving a different problem to Uni :)
samcv hah yeah 16:21
b2gills I just can't help from thinking about the Uni-dome whenever I see Uni (The University of Northern Iowa has a football (rugby) field inside of a dome that used to be an inflated dome) 16:25
16:41 Ven`` joined 16:42 stmuk_ joined 16:47 stmuk_ joined 17:02 stmuk_ joined 17:09 stmuk_ joined 17:19 stmuk joined 17:22 stmuk__ joined 17:35 stmuk joined 18:01 [TuxCM] joined
dogbert17 m: say (^50).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List 18:16
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 m: say (^50).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List 18:17
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 m: say (^50).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 sigh
m: say (^40).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List 18:22
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 m: say (^40).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 m: say (^40).race(batch=>1).map({ sprintf '%1$s %2$s', 5, 42 }).List
camelia (5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42 5 42)
dogbert17 it has a tendency to crash on my system
18:27 AlexDaniel joined
lizmat dogbert17: crash as in how? segfault ? or something less panicky ? 18:36
dogbert17 lizmat: Your printf-style directives specify 0 arguments, but 2 arguments were supplied 18:45
lizmat yuck, that smells like memory corruption
so it lost the format, apparently? 18:46
I wonder if we can make the error tell what the format was
dogbert17 yes, difficult to repro though
lizmat: if you run it with MVM_SPESH_NODELAY=1 and MVM_SPESH_BLOCKING=1 it fails often 18:51
dogbert17 wonders what might be going on 18:56
[Tux] feels somewhat guilty for dragging samcv deeper into this never-ending vortex
nine Unrandomizing hash keys makes the precomp issue go away 18:57
lizmat nine: ok, so we now know what's causing it 19:00
nine And I guess, we'd just have to add some sorting to MVMHash's serialize function 19:02
Of course, this being C code the job is not really straight forward :) 19:04
lizmat nine: it doesn't have to be sorted, but just repeatable ?
so maybe use the sha of the file as the seed for that hash ? 19:05
lizmat waves a bit
nine That would be the build system fix. But adding some sort here is a very localized fix that could fix the problem in general while keeping all the advantages. 19:07
lizmat or perhaps not keep things in a has that need serializing ? 19:11
dogbert17 the race/sprintf thingy looks like some kind of spesh related bug 19:13
nine lizmat: %*GLOBALISH is one of the things getting serialized
19:13 cog_ joined
lizmat ah... hmmm... 19:13
nine Or maybe it's not hitting this serialize function after all... 19:23
Which shouldn't come as a surprise considering that MoarVM has only been able to serialize hashes for a year now 19:28
Ah, there's another function for serializing hashes 19:37
samcv [Tux]: it's fun down here though :) 19:41
nine: where does the serialization code create its hash? 19:42
ah i found it 19:47
nine I have a patch that greatly reduces those differences! 20:02
Pushed it into the make_builds_reproducible_again branch in MoarVM 20:05
It helps, but there's still some hash order issue left.
20:08 Kaiepi joined 20:12 Kaiepi joined 20:13 softmoth joined 20:40 softmoth joined 20:48 softmoth joined 20:49 brrt joined
lizmat hmmm... there I though that $?SUB was a thing (like __SUB__ in Perl 5) 20:59
*thought
jnthn &?ROUTINE I think 21:00
lizmat jnthn++ :-) 21:02
21:39 stmuk_ joined 21:44 stmuk joined 21:47 MasterDuke joined 22:05 stmuk joined 22:16 stmuk_ joined 22:18 stmuk joined 22:26 Kaiepi joined 22:33 cog_ joined 22:55 Kaiepi joined 23:01 skids joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!