AlexDaniel bisectable6oops: run(:out, "ls").out.encoding.say 03:05
bisectable6oops AlexDaniel, Exit code is 1 on both starting points (good=2015.12 bad=b771f1e), bisecting by using the output
AlexDaniel, bisect log: gist.github.com/9f081f1cb9018d60e4...c13609c661
AlexDaniel, (2016-08-26) github.com/rakudo/rakudo/commit/87887da
AlexDaniel lizmat: ↑
lizmat: “concatenate requires a concrete string, but got null” is probably worse than the previous error 03:06
lizmat: I am actually seeing a regression in this case: run(:out, "ls").out.split(0.chr, :skip-empty) 03:08
lizmat: but the regression itself seems to be associated more with github.com/rakudo/rakudo/commit/1a03efe 03:09
bisectable6oops: run(:out, "ls").out.binary.say 03:10
bisectable6oops AlexDaniel, Exit code is 1 on both starting points (good=2015.12 bad=b771f1e), bisecting by using the output
AlexDaniel, bisect log: gist.github.com/3be864fc5825f769c6...b8a6760eca
AlexDaniel, (2016-08-26) github.com/rakudo/rakudo/commit/1a03efe
AlexDaniel lizmat: yeah, so 1a03efe added a check using .binary (which is broken), which made the whole thing blow up 03:11
lizmat: one possible way to fix it is to revert 1a03efe so that it works with non-binary handles 03:12
lizmat: here is a quick ticket just in case: RT #129296 03:20
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129296
AlexDaniel bisectable++ (poor creature, this regression actually broke it) 03:25
Zoffix .tell jnthn the multi == hang is due to inlining. It inlines it, then tries to optimize the inlined bit, and so on. ATM it's over my head on how and if to fix it. I posted details on the ticket: rt.perl.org/Ticket/Display.html?id...xn-1423789 03:57
yoleaux2 Zoffix: I'll pass your message to jnthn.
dalek kudo/nom: dad57b0 | MasterDuke17++ | src/core/IO/Handle.pm:
Set the default encoding for IO::Handle to 'utf8'

When an IO::Handle (or IO::Pipe in the case that prompted the RT) is created the default encoding isn't set. The default is set when .open() is called, but not .new(), this commit fixes that.
Fixes RT #129296
07:20
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129296
dalek kudo/nom: 86d9e91 | niner++ | src/core/IO/Handle.pm:
Merge pull request #882 from MasterDuke17/RT129296

363a3a8 | lizmat++ | src/core/IO/Handle.pm: Revert "Temporary fix for .comb/.split on binary handles"
This reverts commit 1a03efe4e3b61a07b7df505af5e2575893677a8f.
  See irclog.perlgeek.de/perl6-dev/2016-0...i_13230226
Since everything related to encoding is going to be refactored soon anyway, I have no problem reverting this patch (although I think it is the right thing to do in the end).
travis-ci Rakudo build errored. niner 'Merge pull request #882 from MasterDuke17/RT129296 08:18
travis-ci.org/rakudo/rakudo/builds/160784480 github.com/rakudo/rakudo/compare/3...d9e9123039
nine Ok, I've decided to give up on speeding up copying of hashes for now. Even if the API didn't make it as hard, perl's behavior of first getting the list of all keys and only then to start fetching the corresponding values would be enough of a deal breaker. 08:36
Would be lovely to find a way to detect that a hash won't be accessed in Perl 6 code anymore (e.g. it's only passed to a Perl 5 function and not stored anywhere) in which case it would be better to just copy it once. 08:42
t/04-nativecall/12-sizeof.t .............. Failed 6/9 subtests 09:02
travis-ci.org/rakudo/rakudo/jobs/160784481
travis-ci Rakudo build failed. Elizabeth Mattijsen 'Revert "Temporary fix for .comb/.split on binary handles" 10:55
travis-ci.org/rakudo/rakudo/builds/160800879 github.com/rakudo/rakudo/compare/8...3a3a899f2c
nine [Tux]: with github.com/niner/Inline-Perl5/commit/fcc97c0691 Inline::Perl5 should be back in the performance race :) 12:07
[Tux] compiling 12:10
This is Rakudo version 2016.09-7-g363a3a8 built on MoarVM version 2016.09 12:20
csv-ip5xs 9.002
test 15.807
test-t 6.963
csv-parser 17.008
nine That's...not much better :/ Which is odd, since it runs in 5.7 seconds here (assuming Inline::Perl5 is precompiled) 12:22
lizmat [Tux]: maybe you should run the benchmark at least twice to reduce any pre-comp overhead from a first time after a re-compile? 12:25
nine With compilation it's ~ 11s here 12:26
MasterDuke nine: re travis error after you merged my PR, that test always passes (after running several times) on my machine, and i'm not sure why my commit would have effected anything nativecall related 12:28
oh, i see the same test failed on lizmat's last commit 12:29
timotimo did have a commit to NativeCall.pm, but i don't know if it's related 12:31
Zoffix That test has been flopping for ages 12:33
Can't reproduce the failure on my box either :/
Zoffix looks up more
Oh, 12-sizeof.t... I meant the union one
MasterDuke yeah, i just noticed timotimo's travis error was for that, which makes the error for the other commits afterward kind of odd 12:34
timotimo did my commit break some things? 12:35
nine FWIW with rakudo 363a3a8 I'm now at 6.0s. Was 5.7s with 0cf7128 12:37
timotimo i haven't checked how expensive the lock is for setting up natively called routines 12:39
i expected it to be cheap enough in the common case, and not hit nearly often enough to matter :S 12:48
nine Inline::Perl5 has 63 of them 12:49
Though isn't that run only during compilation?
timotimo no, when the routine is first used 12:50
Zoffix finally has a chance to play around with coverage and is following timotimo's instructions on irclog.perlgeek.de/perl6/2016-07-15#i_12846583 12:51
timotimo yay 12:52
Zoffix timotimo, "the crapton of files you end up with...." um, where would they be? I ran MVM_COVERAGE_LOG="coverage-report-%d" make stresstest; but don't see any extra files 13:05
That's with line_based_coverage_4
that I checked out in nqp/MoarVM, then did perl Configure.pl; make; make install;
timotimo should be in your working dir 13:06
Zoffix Nothing found gist.github.com/zoffixznet/c9eada7...151cc7f544 13:07
timotimo gimme a sec.
Zoffix Forgot the asterisks in find, but still no coverage files: gist.github.com/zoffixznet/d1e7e75...e639f8883e 13:08
timotimo hm. maybe my code is dumb about when the %d is at the very very end? 13:09
it *should* be fine, though
does it create a file when you run just perl6 -e '' ?
with the env var set, i mean
Zoffix nope gist.github.com/zoffixznet/6b98ed4...008d20de32 13:12
oh wait
The moar ./perl6 uses is not the updated moar: gist.github.com/zoffixznet/7382637...a9969e6ddc 13:13
I guess I need make install in rakudo's dir too
timotimo i was afraid that might be the case
oh
you didn't give the configure.pl script a --prefix probably 13:14
Zoffix no, I didn't
timotimo yeah
that means you have a moarvm under rakudo/nqp/moarvm/install
but the scripts you have will look in rakudo/install
Zoffix k, now I see the coverage file. Thanks 13:17
timotimo good
timotimo is not a friend of --gen-nqp
[Tux] lizmat, if you take a look at time.pl, you can see that I run a simple short run before the real timeing test to make up for that 14:12
second run with same perl6: 14:13
csv-ip5xs 8.983
test-t 6.929
Zoffix m: say gethostname 14:23
camelia rakudo-moar 363a3a: OUTPUT«camelia␤»
Zoffix I wonder how many uncovered things are uncovered 'cause they weren't meant for end users :) 14:24
timotimo Rakudo::Internals is somewhat new 14:26
nine Where do we need gethostname internally? 14:27
Zoffix It's not used anywhere
timotimo hah
Zoffix Seems it was intended for users, but OS.pm just has that single function.
gist.github.com/zoffixznet/42dc4c6...e5227d18f5
timotimo 2013.11 %) 14:29
nine With github.com/niner/Inline-Perl5/commit/51ec79dd1a I'm now down to 5.5s for csv-ip5xs with current rakudo. 14:38
We now do only 300K native calls for parsing 50K lines of CSV. Was 700K native calls before my array optimizations.
Zoffix \o/ 14:40
dalek ast: 17d9189 | (Zoffix Znet)++ | S02-types/sethash.t:
[coverage] SetHash: .SetHash, .antipairs
14:46
ast: 0edbecb | (Zoffix Znet)++ | S02-types/sethash.t:
Do not rely on order of .antipairs
14:48
Zoffix *sigh*. I need more coffee 14:49
dalek ast: 0cd034f | (Zoffix Znet)++ | S02-types/sethash.t:
Remove extraneous code
timotimo Zoffix: are you making extra sure that the coverage results you're basing your test-writing on are actually accurate? because the coverage report code generates some garbage :( 14:51
Zoffix timotimo, how can I make sure? 14:52
timotimo see if any tests for the method you're looking at already exist, i suppose :\ 14:57
Zoffix What would be a probability of coverage saying a method is uncovered yet is is? 14:58
It did pick up my commits above, saying those two are now covered
timotimo i don't know what exactly causes the report to be bogus 14:59
Zoffix hm
timotimo otherwise i'd be fixing it :)
Zoffix :)
nine And down to 5.0s (of which 4.1s are spent at runtime) with github.com/niner/Inline-Perl5/commit/83746454ce 15:04
Btw. it's 10K lines of CSV. The number it prints is the number of fields parsed. 15:05
So we're now at 18 native calls per line.
The Dancer2 benchmark shed a second, too and is now at 10s (was 11). 15:07
timotimo how big is the dancer2 benchmark's workload?
nine It's metacpan.org/pod/distribution/Danc...l-together translated directly to Perl 6 rendering / (which lists the db entries) 1000 times with 6 entries in the database 15:09
timotimo meaning it also uses DBI from perl5? 15:20
nine yes
timotimo neat. 15:21
nine niner.name/talks/Perl%205%20and%20P.../Dancr.pm6 15:24
Benchmark is: niner.name/talks/Perl%205%20and%20P...-bench.pl6
For comparison: the pure Perl 5 version takes 1.44s 15:30
timotimo wow, less than 10x of a hit? 15:32
nine Actually, yes. And that's with a lot of time wasted by being slow on copying hashes 15:34
timotimo impressive 15:36
nine Oooh...there is a way out! Maps are immutable. So when we pass a Map to Perl 5 code, we can safely copy it right there. This way the user has a choice between copying and wrapping. 15:42
With that I get it down to 7.0s (4.9x slowdown).
timotimo nice! 15:57
DrForr nine: I'm putting together a TT2 clone that should be a drop-in replacement for preexisting Dancer2 templates. 16:00
(working on it when I get bored of figuring out where WS has gone *this* time around in a '= $foo'.) 16:01
dalek ast: b0b1ead | (Zoffix Znet)++ | S32-list/classify.t:
Strip trailing whitespace
16:08
ast: 0f4199b | (Zoffix Znet)++ | S32-list/c (2 files):
[coverage] Any: .classify/.categorize with bad arguments throw
16:14
Zoffix Were coverage report 100% accurate, roast covers 63.4% of code :) 16:20
timotimo that's really not alot 16:24
brave sir not-a-lot
mst 'not alot' isn't better than you at anything? 16:44
I suppose how negation affects it depends on whether alot is an individual symbol or a junction
timotimo that's right 16:49
arnsholt Mid 60% isn't too shabby, I think 16:51
If memory serves, the project my friend (and now colleague) works on has something like 30-ish percent coverage 16:52
On a big Java EE project
nine Coverage percentages are not comparable at all unless you define exactly what they mean. 16:54
Is it % of functions, lines, expressions, sub-expressions, ops?
Measuring Perl 5 code with Devel::Cover you usually get the first 30 % by running anything at all. 16:55
arnsholt True, true 17:04
Can't remember what they're measuring
timotimo we currently measure lines that appear in some frame somewhere vs lines that have definitely been hit 17:08
with some uncertainty what makes some of these darn lines show up in the moarvm dump but not be hit when run
ugexe as a performance data point zef installs itself in 30 seconds (2016.09) vs 90 seconds (2016.01) 17:32
tadzik niiice
japhb Oh, that is huge. 17:33
Zoffix m: multi foo ($count) { say "Not right one" }; multi foo (*@_ [&first, *@rest]) { say "Right one" }; foo [{;}]; 18:39
camelia rakudo-moar 363a3a: OUTPUT«Not right one␤»
Zoffix m: multi foo ($count) { say "Not right one" }; multi foo (@_ [&first, *@rest]) { say "Right one" }; foo [{;}]; 18:40
camelia rakudo-moar 363a3a: OUTPUT«Right one␤»
Zoffix Well, shit. Who knew writing coverage would find a bug so fast :)
The bug being this .tree multi being called, instead of the one below it: github.com/rakudo/rakudo/blob/363a...ny.pm#L166 18:41
timotimo wow! 18:43
i did *not* expect that
Zoffix m: multi foo (Int $count) { say "Not right one" }; multi foo (*@_ [&first, *@rest]) { say "Right one" }; foo [{;}];
camelia rakudo-moar 363a3a: OUTPUT«Right one␤»
Zoffix m: multi foo (Int(Cool) $count) { say "Not right one" }; multi foo (*@_ [&first, *@rest]) { say "Right one" }; foo [{;}]; 18:44
camelia rakudo-moar 363a3a: OUTPUT«Not right one␤»
Zoffix m: say [{;}] ~~ Cool
camelia rakudo-moar 363a3a: OUTPUT«True␤»
Zoffix really :S
m: say [{;}].^mro
camelia rakudo-moar 363a3a: OUTPUT«((Array) (List) (Cool) (Any) (Mu))␤»
Zoffix Ah
dalek kudo/nom: 96299c7 | (Zoffix Znet)++ | docs/ChangeLog:
Start 2016.10 section
18:59
kudo/nom: 34f9508 | (Zoffix Znet)++ | / (2 files):
Fix .tree([&first]) incorrectly calling .tree($count) candidate

We can't make tree(Any:D: @ [&first, *@rest]) candidate a *@_ slurpy, becauses lists are Cool and then the tree(Any:D: Int(Cool) $count) candidate becomes narrower.
Fix by adding another multi for (&first, *@rest) and removing slurpy from @_
19:08
Zoffix s/@_/@/
dalek ast: a176e08 | (Zoffix Znet)++ | S02-lists/tree.t:
[coverage] Cover all branches of .tree
19:09
Zoffix Hm. Another one. sorta-not-really-a-bug tho. 19:20
s: Array, 'append', \(42)
SourceBaby Zoffix, Something's wrong: ␤ERR: Unhandled exception: Missing or wrong version of dependency 'src/Perl6/Grammar.nqp' (from 'gen/moar/m-main.nqp')␤ at <unknown>:1 (/home/zoffix/services/sourceable/building-perl6/perl6.moarvm:<dependencies+deserialize>)␤
Zoffix s: Array, 'append', \(42)
SourceBaby Zoffix, Something's wrong: ␤ERR: Unhandled exception: Missing or wrong version of dependency 'src/Perl6/Grammar.nqp' (from 'gen/moar/m-main.nqp')␤ at <unknown>:1 (/home/zoffix/services/sourceable/building-perl6/perl6.moarvm:<dependencies+deserialize>)␤
Zoffix sigh
That's NOT that bug :P
This is the candidate that would be called: github.com/rakudo/rakudo/blob/363a...ny.pm#L186 19:21
But:
m: say Array.append: 42
camelia rakudo-moar 34f950: OUTPUT«Cannot resolve caller STORE(Array: Array); none of these signatures match:␤ (Array:D $: Iterable:D \iterable, *%_)␤ (Array:D $: Mu \item, *%_)␤ in block <unit> at <tmp> line 1␤␤»
Zoffix m: use nqp; say nqp::istype(Array,Positional) 19:22
camelia rakudo-moar 34f950: OUTPUT«1␤»
Zoffix m: class { method append (|c) { dd c} }.append: 42 19:23
camelia rakudo-moar 34f950: OUTPUT«\(42)␤»
Zoffix doesn't get why one works but not the other. 19:24
Since condition is SELF = nqp::istype(SELF,Positional) ?? SELF.new !! Array.new;
m: say Int.append: 42 19:27
camelia rakudo-moar 34f950: OUTPUT«Cannot modify an immutable Int␤ in block <unit> at <tmp> line 1␤␤»
Zoffix Ah. I see (I think)
Zoffix tries a fix
Yup. Fixed. 19:31
timotimo++ damn this coverage thing is useful. I found a nest of bugs just now :) 19:40
timotimo <3 19:42
i'm super glad someone's taking advantage of it
nine I guess the lesson of this year is: tools are everything 19:44
lizmat [Tux]: message received :-) 19:46
timotimo i could see that
travis-ci Rakudo build passed. Zoffix Znet 'Start 2016.10 section' 19:53
travis-ci.org/rakudo/rakudo/builds/160872898 github.com/rakudo/rakudo/compare/3...299c745398
Zoffix s/nest of bugs/no bugs/; :) 20:18
The Any:U.[push|append|prepend|unshift] are done that way for autovivification purposes
They don't show up as covered, even though there are definitely tests calling it (since I broke them now :P) 20:19
timotimo yeah, the coverer isn't good enough :( 20:24
travis-ci Rakudo build passed. Zoffix Znet 'Fix .tree([&first]) incorrectly calling .tree($count) candidate 20:25
travis-ci.org/rakudo/rakudo/builds/160874432 github.com/rakudo/rakudo/compare/9...f95082e939
timotimo hm. i wonder if it b0rks because of static inlining via the rakudo optimizer?
Zoffix maybe 20:29
timotimo theoretically you could build a core setting with --optimize=off 20:30
Zoffix I'll try tomorrow or later tonight 20:31
timotimo cool 20:32
dalek ast: e427332 | (Zoffix Znet)++ | S (3 files):
[coverage] cover all non-CAPS methods in Any.pm
20:33
kudo/nom: 8991360 | (Zoffix Znet)++ | t/spectest.data:
Add t/spec/S29-any/deg-trans.t

Degenerate and Transformative Any methods
AlexDaniel dalek: ;) 20:56
Zoffix cc1: error: argument to ‘-O’ should be a non-negative integer, ‘g’, ‘s’ or ‘fast’ 20:57
Zoffix goes with --no-optimize
timotimo er 20:58
no
wrong position to put that argument
Zoffix: it needs to go in the long-ass commandline that generates CORE.setting.moarvm
Zoffix What level is that? MoarVM? NQP? Rakudo? 20:59
timotimo rakudo
in its "make" step
the one that has "stage parse" and all that
Zoffix k
timotimo, I think this is it. The old make stresstest shows `multi method append(Any:U \SELF: |values)` as uncovered. I built with optimize=off and ran report for `./perl6 -e 'my %h; say %h<c><d>.append: 42;'` (which calls that .append under the hood) and now it's showing it as covered 21:10
Gonna generate a new report for the full stresstest now
timotimo a-ha! 21:17
damn, i thought it ought to have transferred over the nodes so it'd generate the line numbers properly
but nooo
dalek kudo/nom: 9b6f2eb | lizmat++ | src/core/List.pm:
Fix for RT #129299

The List.iterator assumed that a List could never have a "hole" in it. This allowed the iterator to not have to check the index against the number of elements in the list. I have now reinstated that check, albeit only when it finds what could be a hole. If the index is then still less than the number of elems, we return the hole and continue.
This adds an extra test for *each* List.iterator. Going to think about another way to fix this if this shows up in reduced performance somehow.
21:27
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129299
lizmat good night, #perl6-dev! 21:29
timotimo gniet liz!
Zoffix night 21:30
hm weird 21:34
timotimo, now it's showing it as uncovered again :/ That's on a full stresstest run
timotimo did you delete reports in between doing the stuff?
Zoffix Yeah 21:35
timotimo also, if the script that creates the html files is too slow, do you run grep for CORE and sort and uniq over it?
Zoffix This is basically what I ran: gist.github.com/zoffixznet/fcc1fe0...132986f916 21:37
With `MVM_COVERAGE_LOG='coverage/cover-%d' make stresstest` being `MVM_COVERAGE_LOG='coverage/cover-%d' ./perl6 -e 'my %h; say %h<c><d>.append: 42;'` when it did show `multi method append(Any:U \SELF: |values)` as covered
timotimo you can run the stresstest with TEST_JOBS btw 21:38
that's why i introduced the support for %d in the filename
Zoffix I have it in .bashrc
timotimo OK!
yeah that looks like the right way to invoke that
Zoffix tries the `my %h; say %h<c><d>.append: 42;'` one again 21:39
Yup. Now it shows it as covered :} I also ran make t/spec/S02-types/autovivification.t (that contains that piece of code) and it told me it's uncovered now :S 21:44
Even though multi method push(Any:U \SELF: |values) and multi method unshift(Any:U \SELF: |values) do show up as covered, even though the full stresstest shows them as uncovered 21:45
Oh god... I found the problem 21:53
<--- that's the problem :)
The test file doesn't have .append in it. Only .push and .unshift that do show up covered :} 21:54
timotimo ooooh! 21:58
dalek ast: 510b44a | (Zoffix Znet)++ | S02-types/autovivification.t:
Add autovivification tests for append/prepend
22:38
Zoffix m: "ac" ~~ / (a) (b)? (c) /; say $/.list 23:23
camelia rakudo-moar 641ee1: OUTPUT«(「a」 (Mu) 「c」)␤»
Zoffix star: "ac" ~~ / (a) (b)? (c) /; say $/.list
camelia star-m 2016.04: OUTPUT«(「a」 (Any) 「c」)␤»
Zoffix committable6, 2016.07 m: "ac" ~~ / (a) (b)? (c) /; say $/.list 23:24
committable6 Zoffix, ¦«2016.07»: (「a」)
timotimo Zoffix: was that also a [coverage] commit? :) 23:26
Zoffix Yeah, I guess.
m: my @a = [1, 2, 3]; @a[1]:delete; dd @a 23:29
camelia rakudo-moar 641ee1: OUTPUT«Array @a = [1, Any, 3]␤»
Zoffix m: "ac" ~~ / (a) (b)? (c) /; dd $/.list
camelia rakudo-moar 641ee1: OUTPUT«(Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 1, from => 0), Mu, Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 2, from => 1))␤»
Zoffix .ask lizmat is it correct that 9b6f2eb543 makes "ac" ~~ / (a) (b)? (c) /; $/.list has a Mu in the hole and not an Any? In 2016.04 it used to be an Any and if you do my @a = [1, 2, 3]; @a[1]:delete; dd @a you get an Any in the hole 23:30
yoleaux2 Zoffix: I'll pass your message to lizmat.
timotimo it makes me quite happy that the coverage thing gets to help
Zoffix Yeah, it's useful.
Not everything can be covered, right? Like ternaries for example. I recall some sort of limitation like that in P5. Like these i.imgur.com/ZUqwmh9.png as the tests for them are there: github.com/perl6/roast/blob/master...tree.t#L36 23:35
timotimo the coverage tool knows nothing about conditions or control flow or anything like that 23:39
it just says "hey, there's a line number here! *plop*" 23:40
Zoffix Ah
m: use Test; "ac" ~~ / (a) (b)? (c) /; is-deeply $/.list, (Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 1, from => 0), Mu, Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 2, from => 1)) 23:53
camelia rakudo-moar 641ee1: OUTPUT«Type check failed in binding to <anon>; expected Any but got Mu (Mu)␤ in sub is-deeply at /home/camelia/rakudo-m-inst-1/share/perl6/sources/C712FE6969F786C9380D643DF17E85D06868219E (Test) line 484␤ in block <unit> at <tmp> line 1␤␤»
Zoffix m: "ac" ~~ / (a) (b)? (c) /; $ = $/.list eqv (Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 1, from => 0), Any, Match.new(ast => Any, list => (), hash => Map.new(()), orig => "ac", to => 2, from => 1)) 23:56
camelia rakudo-moar 641ee1: OUTPUT«Type check failed in binding to <anon>; expected Any but got Mu (Mu)␤ in block <unit> at <tmp> line 1␤␤»
Zoffix I think it's the same thing as === that I wanted to fix but failed, 'cause the fix was breaking Junctions.