Perl 6 language and compiler development | Logs at colabti.org/irclogger/irclogger_log/perl6-dev | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
Set by Zoffix on 27 July 2018.
timotimo we're not testing. this is serious. 00:00
i didn't time how much exactly it took while i was profiling, but it's spending like 4 seconds in GC 00:06
i see it taking a whole load of closures 00:07
if zef didn't use "exit", the profiler might not get quite as confused 00:11
MAIN_HELPER MAIN run-script load load load load load <anon> 00:12
weird %)
timotimo the main moar process calls stat 1163 times, fun. 00:24
AlexDaniel timotimo: that doesn't sound like much? 00:25
timotimo looks like every file gets statted four times, at least the ones under share/perl6/site/dist/ 00:29
it also stats a whole bunch of precomp files 00:30
AlexDaniel isn't it normal?
I mean if you run something like zef --help, wouldn't it stat a bunch of stuff also? 00:31
just to see if zef and all of its modules are precompiled right and stuff like that
timotimo i don't actually know enough about that 00:33
AlexDaniel but anyway, why calling stat 1000 times is a problem?
I expect any 1000 of filesystem operations to take absolutely no time 00:34
timotimo yeah, just like 2 seconds, so it's not our culprit at least
AlexDaniel and by no time I mean in context of 10 seconds that I see
timotimo er, 8 seconds?
no, those must be msecs 00:35
. is not thousands separator %)
i'd really like to know who calls exit there 00:38
samcv [Tux]: when you're around pm me about utf16 stuff 00:39
AlexDaniel timotimo: github.com/ugexe/zef/blob/c07448cd...I.pm6#L183 00:40
timotimo: you can comment it out 00:41
timotimo TYVM
AlexDaniel hm, interesting 00:44
so returning something from MAIN doesn't use that return value as exit code
timotimo grr, it's still calling exit somehow 00:45
ah, abort also exits
AlexDaniel so you have to use `exit` to abruptly end your program just so that you can have the exit code that you need
timotimo IMO, it shouldn't exit with what you return 00:46
it should exit 0 if it exits fine and something else if you throw
AlexDaniel throw what? 00:47
I want to exit with exit code 5, how?
without throwing LEAVE and other cleanup into a dumpster
timotimo that's when you'd exit 5
AlexDaniel well that sucks
timotimo hmm.
ISTR some way to give an exception an exit code along with it
was that a thing? 00:48
you can teach exit what to do
m: sub test { LEAVE say "exited test"; blorp }; sub blorp { LEAVE say "exited blorp"; exit 5 }; test
camelia ( no output )
timotimo mhm
AlexDaniel I mean, someone will probably argue that the solution to this is to have no LEAVE blocks in your MAIN 00:49
and instead have them somewhere else
timotimo something is *still* calling exit
AlexDaniel m: sub real-MAIN-right-here { LEAVE say ‘foo’; 2 + 2 }; sub MAIN { real-MAIN-right-here; exit 5 } 00:50
camelia foo
AlexDaniel timotimo: are you sure you're using the modified file? 00:51
timotimo i have sensible timings!! 00:52
AlexDaniel \o/
MasterDuke timotimo: heh, couldn't you use your profiler output to tell you where exit was being called from? 00:53
timotimo i was 00:54
i see 4 seconds of json parsing
if i see it correctly, parsing 150 json documents
AlexDaniel timotimo: using what? JSON::Fast?
timotimo no, rakudo's built-in one 00:55
700ms of Zef::Identity parsing
AlexDaniel hm, what are these 150 json files…
if it was 1000 I'd understand 00:56
timotimo debugserver should be able to show me 00:57
but i'd probably like to go to bed now
do you want my sql file to look at it with the profiler frontend?
AlexDaniel I'm going to bed now too
timotimo SGTM
rest well!
AlexDaniel but
timotimo you may have to reactivate me tomorrow
AlexDaniel github.com/ugexe/zef/blob/master/l...ty.pm6#L10
this just needs to be a .split(‘:’) or something like that 00:58
MasterDuke timotimo: can you put your sql file somewhere?
timotimo sure 00:59
timotimo hack.p6c.org/~timo/zefnoop.sqlite3 00:59
4dd64cffb0a4d8a8ce789fb015048951056586fe153a8d044964aa8beee191e4 /tmp/zefnoop.sql
sha256sum
AlexDaniel or maybe just optimizing the grammar should do 00:59
token name { <token>+ }; token token { <-restricted> }; token restricted { < : > } 01:00
timotimo i don't know if it's that grammar or the other one
AlexDaniel all these hoops to jump through just to match anything but :
timotimo that is taking time
$id !~~ /':ver' | ':auth' | ':api' | ':from'/ 01:01
no clue if that path is taken at all anyway
AlexDaniel 💤 01:04
samcv i just posted this github.com/MoarVM/MoarVM/issues/966 a request for discussion about how our utf16 works. since it basically has many answered questions since we only supported Str.encode or Buf.decode, and not reading a file handle or writing to one 05:56
maybe i should post it in the perl6 issue tracker too? 05:57
lizmat Files=1252, Tests=76124, 334 wallclock secs (15.42 usr 5.00 sys + 2366.98 cusr 216.42 csys = 2603.82 CPU) 08:02
lizmat .tell [Tux] github.com/MoarVM/MoarVM/issues/966 08:14
yoleaux lizmat: I'll pass your message to [Tux].
Geth rakudo: a4db33e72b | (Elizabeth Mattijsen)++ | 6 files
Remove EmptySeq, in response to R#2281

This actually, surprisingly, reduces the .moarvm file by 2.5K		 08:41
synopsebot R#2281 [open]: github.com/rakudo/rakudo/issues/2281 [6.d review] Problems with `EmptySeq`
Geth roast: 6360992678 | (Elizabeth Mattijsen)++ | S32-io/io-spec-unix.t
Unfudge test fudged for R#2281
 08:45
synopsebot R#2281 [closed]: github.com/rakudo/rakudo/issues/2281 [6.d review] Problems with `EmptySeq`
[Tux] Rakudo version 2018.08-108-ga4db33e72 - MoarVM version 2018.08-81-gff0fb63a4
csv-ip5xs0.916 - 0.927
csv-ip5xs-207.158 - 7.630
csv-parser23.135 - 24.424
csv-test-xs-200.426 - 0.432
test8.855 - 9.024
test-t2.072 - 2.170
test-t --race0.862 - 0.922
test-t-2037.081 - 39.067
test-t-20 --race11.197 - 14.179
 09:12
yoleaux 08:14Z <lizmat> [Tux]: github.com/MoarVM/MoarVM/issues/966
samcv o/ [Tux] 09:28
nine Oh boy, NQP defaulting to num sucks endlessly... 09:57
timotimo yep 09:58
did you see the branch to make it default to int?
nine Yes, some time 09:59
So how do I tell it that in @static_lex_values[4 * $i] the index ought to be an int?
Getting MVMArray: atpos expected int register
timotimo use nqp::atpos instead :x
oh 10:00
or nqp::mul_i(4, $i)
or put the in-between into an int variable
nine the int variable trick doesn't seem to work
timotimo fucking hell.
nine mul_i doesn't work 10:01
atpos_i is good
timotimo oh, but that's about whether you have an int array or an object array, not about the keys 10:30
hmm.
nine That's odd, because nqp::atpos does not work 10:33
lizmat you where talking about problems with the index, no? 10:38
nine That's what I assumed
lizmat if so, nqp::atpos or nqp::atpos_i shouldn't make a difference as the suffix is about the *target*, no
/
?
nine According to my understanding. But maybe the error message is just misleading 10:40
timotimo nqp: my @a := nqp::list_i(1, 2, 3); say(nqp::atpos(@a, 1)) 10:49
camelia MVMArray: atpos expected int register
at <tmp>:1 (<ephemeral file>:<mainline>)
from gen/moar/stage2/NQPHLL.nqp:1675 (/home/camelia/rakudo-m-inst-1/share/nqp/lib/NQPHLL.moarvm:eval)
from gen/moar/stage2/NQPHLL.nqp:1914 (/home/camelia/rakudo-m-inst-1…
timotimo nqp: my @a := nqp::list_i(1, 2, 3); say(nqp::atpos_i(@a, 1))
camelia 2
timotimo this is what i'm refering to
nine And indeed: @!static_lex_values := nqp::list_i(); 10:50
timotimo oh 10:51
your error message was this exactly
m(
i've given you the wrong advice completely
nine Darn...the operands from op_info does not seem to be available in NQP 10:56
timotimo they aren't in that module you include with most of the info? 10:57
nine Ah, I guess that's @MAST::Ops::values 10:59
And I guess @MAST::Ops::offsets tells me where in that value list an op's arguments start 11:02
lizmat hmmm... do we have a simple way to tell if something is a container ? 11:04
*without* using NQP ?
AlexDaniel timotimo: any news? 11:05
lizmat if not, how about adding a Mu::CONTAINER(\a) { nqp::hllbool(nqp::iscont(a)) } ?
or rather:
if not, how about adding a Mu.CONTAINER(\SELF:) { nqp::hllbool(nqp::iscont(SELF)) } ?
AlexDaniel nine: btw have you seen this discussion? Starting around here and going all the way to the next day: colabti.org/irclogger/irclogger_log...09-15#l540 11:06
lizmat sub is-container(\it) { it.VAR.^name ne it.^name } # feels rather hacky and sub-performant 11:07
AlexDaniel TL;DR `time zef --dry --/test --/build install JSON::Fast` takes 10 seconds or maybe a bit less depending on the system
nine AlexDaniel: no, missed that apparently 11:08
150 JSON files? 11:11
AlexDaniel well, at least 150 calls to from-json I guess
that's as far as we managed to get yesterday, I thought maybe you know something :) 11:12
nine Well I was pretty sure that almost all of that waiting time is JSON parsing
AlexDaniel nine: ok, but what kind of json files? 11:13
nine Personally, I'd probably just store that ecosystem info in an SQLite file. But 150 calls sounds more like it's parsing JSON of installed dists, too 11:14
Geth roast: 95bb2232ee | (Zoffix Znet)++ | S16-io/words.t
[v6.d REVIEW] Precisify :close on words

  - Spec partial consumption of iterator leaves the handle in
   open state
  - Spec consuming $limit items from iterator closes the handle
   even before we got IterationEnd
Orig: github.com/perl6/roast/commit/b0c4a7a37		 11:20
AlexDaniel hm 11:21
I don't think parsing json is a problem, but the way it does it is
so
I see 0.825388s + 0.63321996s for parsing both ecosystem files with JSON::Fast 11:22
nine Not great, but also not terrible
AlexDaniel if parallelized it'll be under 1 second, not terrible indeed 11:23
but
commenting out `use JSON::Fast`
5.20290931s + 5.636712s
AlexDaniel so Rakudo::Internals::JSON is that bad in terms of speed 11:24
lizmat yeah, afaik it's only there for backward compatibility 11:25
afk&
AlexDaniel makes me wonder if zef can use JSON::Fast if it's available, and fallback to Rakudo::Internals::JSON
I do understand that the two are possibly incompatible, but still…
timotimo how fast do we figure out if something is installed or not? 11:26
i.e. how much does it cost to check for JSON::Fast if it isn't there?
AlexDaniel IIRC relatively fast 11:27
timotimo also, maybe there's a cheap 2x improvement somewhere inside rakudo's json parser
for example, a fast path for strings that have no escapes in them
AlexDaniel timotimo: ah actually, you don't need zef to check that 11:28
nine I sure hope it's pretty damn fast afte all the time I spent optimizing our module loader :)
AlexDaniel a `require` or something like that will do
timotimo mhm
timotimo i'm still on moarvm with --optimize=0, lol 11:29
AlexDaniel timotimo: any reason to hunt for 2x improvements when we can make it more than 6x as fast using JSON::Fast? :) 11:30
that will only affect the initial zef install when JSON::Fast is not available
timotimo hmm.
AlexDaniel once the user installs pretty much… any module… then it's guaranteed they'll have JSON::Fast :) 11:31
timotimo i suppose being one of the most depended-on modules, it's not so bad to support it in zef, too, yeah
AlexDaniel looks at finanalyst.github.io/ModuleCitation/
timotimo yeah 11:32
timotimo feels weird to have made that %) 11:32
Geth roast: ee937b866b | (Zoffix Znet)++ | packages/Test/Util.pm6
Implement `group-of` helper routine

  - Saves a line of typing when writing subtests
  - Ensures you always have a plan
Geth roast: fdf1b1f521 | (Zoffix Znet)++ | S16-io/words.t
Use new group-of helpers
 11:33
nine Speeding up rakudo's JSON parser would also help module installation directly
timotimo it's also used for other stuff?
nine rakudo wouldn't have a JSON parser if it wasn't used in rakudo, would it? :) 11:34
timotimo inertia :P
nine Actually I'm not sure if we parse JSON again during module installation. But it's definitely done for accessing %?RESOURCES 11:35
timotimo so if you want to really slow, have lots of stuff in %?RESOURCES? 11:36
and access it at run time? :)
AlexDaniel but then, if over 35% of modules need it in the end, and if rakudo itself needs a json parser… any reason not to have JSON::Fast in rakudo? :)
nine Stuff in your META data in general
timotimo or, you know, try runtime requiring JSON::Fast for inside usage? 11:37
Geth nqp: 7559d37702 | (Tom Browder)++ (committed using GitHub Web editor) | src/HLL/Compiler.nqp
fix format
 11:49
AlexDaniel tbrowder__: still looks misaligned to me 11:54
tbrowder__: screenshots.firefox.com/pqRlgDX5fQ...github.com
Geth roast: 40ab9e9f3b | (Zoffix Znet)++ | S16-io/lines.t
[v6.d REVIEW] Precisify .lines tests

  - Use is-eqv to spec return type properly
  - Spec pulling $limit items from iterator closes the handle
   (i.e. we don't have to do one more pull for IterationEnd)
Orig: github.com/perl6/roast/commit/f3d1f6798		 12:12
roast: 8c89f86825 | (Zoffix Znet)++ | S16-io/words.t
[v6.d REVIEW] Fix .words test

In earlier commit[^1], we specced that reading $limit elements closes the handle (without pulling once more for IterationEnd), however the commit had an error and used incorrect indices.
Fix to slice only first two items
  [1] github.com/perl6/roast/commit/95bb2232ee		 12:13
roast: c231666dec | (Zoffix Znet)++ | 2 files
Fudge .lines/.words :close $limit tests

R#2283 github.com/rakudo/rakudo/issues/2283		 12:18
synopsebot R#2283 [open]: github.com/rakudo/rakudo/issues/2283 [6.d review][IO] .lines/.words :close closes too late when $limit is specified
tbrowder__ AlexDaniel: in web editor it looked ok, after editing, in. 12:26
AlexDaniel the web editor sucks then
tbrowder__ in my chrome browser on ipad the “my” is hanging out one char to the left 12:27
AlexDaniel the problem is tabs
tbrowder__ yeah, probably, but in orig source or githubs display of it? 12:28
AlexDaniel screenshots.firefox.com/SbtlAZIWuM2DTLxx/null
tbrowder__ weird 12:29
AlexDaniel IMO we shouldn't use tabs in the source code
tbrowder__ i agree, i thought xt did something about that 12:30
Geth ¦ rakudo: lizmat self-assigned .lines/.words :close closes too late when $limit is specified github.com/rakudo/rakudo/issues/2283
AlexDaniel only in doc repo
tbrowder__ afaik, my emacs fix earlier this year stopped tabs from my code 12:31
i’ll try to remember to not edit code with web editor, sorry
AlexDaniel tbrowder__: tabs existed before your change 12:32
it seems like you added some spaces to make it align in the view of the web editor, and the right solution was to remove tabs first and then do that :)
Zoffix Just had a realization: for 6.d release, we'll likely have a Rakudo that passes a lot more tests on *nix than on Windows :/ 12:50
S#320 (most recent failures are at the bottom)
synopsebot S#320 [open]: github.com/perl6/roast/issues/320 Windows roast failures just before 2017.09 release
Zoffix Tons of IO tests, so I'm guessing lots of those are new IO Grant test that might've been written wrong. 12:51
Geth roast: 5b51298828 | (Zoffix Znet)++ | S02-types/baghash.t
[v6.d REVIEW] Don't spec BagHash.Str in tests not about that

Orig: github.com/perl6/roast/commit/8cd62dba8		 13:04
roast: e21d1022d3 | (Zoffix Znet)++ | 2 files
[v6.d REVIEW] Use more appropriate test routines
 13:12
Geth roast: 445c3c50dc | (Zoffix Znet)++ | 2 files
[v6.d REVIEW] Use is-eqv for Seq test in Set[Hash].roll

Orig: github.com/perl6/roast/commit/c4c10d506		 14:02
roast: 539f9bde06 | (Zoffix Znet)++ | 2 files
[v6.d REVIEW] Appendicise Pair.new tests

Orig: github.com/perl6/roast/commit/a523aef60		 14:04
roast: ebd8e2497d | (Zoffix Znet)++ | S16-io/put.t
[v6.d REVIEW] Improve `put` tests

Slip the test values into the routines to test the multi-arg variants as well.
Orig: github.com/perl6/roast/commit/4d0f97a92		 14:08
AlexDaniel squints at github.com/ugexe/zef/blob/846c5ab9...m6#L32-L34 14:24
I'm seeing this: zef thinks that the dependency is installed but module tests fail because the dependency is not installed 14:26
my $ENV := %*ENV; 14:28
hmmmm…
that will mangle the original %*ENV, right?
nine yes 14:29
AlexDaniel whenever $proc.start(:$ENV, :cwd($path)) { $passed = $_.so } 14:30
ahhhhhhhhhhhhh…
I think different :cwd is the issue 14:31
yes 14:34
slapping .absolute on my paths fixed it
will ticket both issues
github.com/ugexe/zef/issues/270 github.com/ugexe/zef/issues/271 14:40
AlexDaniel alright I'm getting somewhere 15:59
timotimo slaps roof of :cwd 16:24
this bad boy can fit so many relative paths in it
tbrowder__ AlexDaniel: i’ll do that next time (remove tabs first)! 17:51
Geth roast: 1564ef710c | (Zoffix Znet)++ | fudgeandrun
[v6.d REVIEW] Warn if we don't find local binary

Otherwise the user might be completely unaware they might be roasting with the wrong bindary
Orig: github.com/perl6/roast/commit/568eb76ae		 18:13
Geth roast: a767b1cfd1 | (Zoffix Znet)++ | S32-io/io-cathandle.t
[v6.d REVIEW] Reword IO::CatHandle.lines(0)

Explicitly say what we expect
Orig: github.com/perl6/roast/commit/6e8a0be41		 18:23
6.d-prep: dffa264241 | (Zoffix Znet)++ (committed using GitHub Web editor) | TODO/README.md
Update spec review

  - Update zofglop commit
  - List QuantHash ops as weakness		 18:24
Geth nqp/nqp-mbc: 8d0dda3083 | (Stefan Seifert)++ | mbc.nqp
Prototype for a mbc writer in nqp
 19:14
roast: f477dbfc7f | (Zoffix Znet)++ | S32-io/pipe.t
[v6.d REVIEW] Actually test slurp content in bin pipes

Orig: github.com/perl6/roast/commit/29582e15c		 19:16
nqp/nqp-mbc: a32e1ba794 | (Stefan Seifert)++ | mbc.nqp
Replace old multi dispatch op writing with type info based infrastructure
 19:31
Geth roast: 72be9066a6 | (Zoffix Znet)++ | S32-io/io-cathandle.t
[v6.d REVIEW] Fix copy-pasta in IO::CatHandle.path

Orig: github.com/perl6/roast/commit/175ba458e		 19:42
Geth roast: e3627013eb | (Zoffix Znet)++ | S32-io/pipe.t
[v6.d REVIEW] Use more exact test for IO::Pipe.proc

Orig: github.com/perl6/roast/commit/47c88abe4		 20:02
roast: 4ff231bbb1 | (Zoffix Znet)++ | S32-io/slurp.t
[v6.d REVIEW] Don't rely on Buf.gist in &slurp test

Orig: github.com/perl6/roast/commit/38c61cd32		 20:07
roast: 7d47580eb8 | (Zoffix Znet)++ | S03-metaops/cross.t
[v6.d REVIEW] Test actual values in `X` with empty RHS

Orig: github.com/perl6/roast/commit/0faf3c354		 20:11
roast: 2bf3198659 | (Zoffix Znet)++ | S03-metaops/cross.t
Fudge crashing X+ test

R#2286 github.com/rakudo/rakudo/issues/2286		 20:12
synopsebot R#2286 [open]: github.com/rakudo/rakudo/issues/2286 `(1,2) X+ ()` crashes
Geth roast: 2cb8e5f134 | (Zoffix Znet)++ | 2 files
[v6.d REVIEW] Appendicize Junction.new with wrong args

Orig: github.com/perl6/roast/commit/beea79813		 23:14
AlexDaniel alright!!! 23:39
it bisected Foo::Regressed to 741ae6f4eda001e5196cd7e156e887e7279cf1d8 :)
awesome
and Foo::Regressed::Very also 23:43
cool, so it works
it also tests for flappers which is helpful 23:44
buuut a full run will have to wait for tomorrow as I'm going to bed now
MasterDuke AlexDaniel++ 23:56
AlexDaniel also I haven't decided what kind of output I want 23:57
initially I was thinking to just use toaster as frontend but this one is a bit different 23:58
sooo… graphviz? 23:59
I have all the info to output a dot file

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!