dalek kudo/nom: 61a71f5 | MasterDuke17++ | src/core/metaops.pm:
Fix RT #118223
 01:35
kudo/nom: beec02a | (Zoffix Znet)++ | src/core/metaops.pm:
Merge pull request #859 from MasterDuke17/nom

Fix RT #118223
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=118223
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=118223
travis-ci Rakudo build failed. Zoffix Znet 'Merge pull request #859 from MasterDuke17/nom 02:49
travis-ci.org/rakudo/rakudo/builds/159764045 github.com/rakudo/rakudo/compare/6...ec02a6fa69
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
[Tux] This is Rakudo version 2016.08.1-184-gbeec02a built on MoarVM version 2016.08-43-g3d04391 05:58
csv-ip5xs 10.830
test 16.131
test-t 6.838
csv-parser 17.228
nine mst: I really thought so. But a quick test shows me wrong. I have no idea how this works and where the INIT block is called. But I guess, I should just be glad. This opens a lot of possibilities :) 06:06
dalek kudo/nom: 5f2e96b | lizmat++ | src/core/Channel.pm:
Make Channel.elems return a Failure

Rather than silently eating the currently available values on the Channel and then returning the number of values eaten.
Inspired by discussion started at
   irclog.perlgeek.de/perl6/2016-09-14#i_13206149		 06:47
travis-ci Rakudo build failed. Elizabeth Mattijsen 'Make Channel.elems return a Failure 07:42
travis-ci.org/rakudo/rakudo/builds/159797732 github.com/rakudo/rakudo/compare/b...2e96b8c046
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
jnthn morning, #perl6-dev 08:58
brrt good morning, jnthn 09:10
jnthn Well, there's the first bug of the day hunted down... 09:53
RT #129249 fwiw 09:54
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129249
jnthn Turns out we mis-built the QAST tree, causing an explosion in the thingy that makes NFAs out of regexes 09:55
DrForr Uh. Looking at the report... 09:59
jnthn m: use Test; my %what = foo => 42, bar => 43; my $m = "foo3bar4" ~~ /$<cat>=@(%what.keys) 4/; is $m, "bar4"; is $m<cat>, "bar"
camelia rakudo-moar 5f2e96: OUTPUT«===SORRY!===␤Cannot find method 'rxtype' on object of type QAST::Op␤»
DrForr Yeah, looking now. I'm right now worried more aobut what's going to change the NQPMatch output. 10:00
Zoffix hm 10:01
DrForr *about
Zoffix Those travis failures are because I fixed the harness yesterday, so now it's correctly detecting the failing test suite. 10:02
dalek kudo/nom: c3b044d | jnthn++ | src/Perl6/Actions.nqp:
Correct code-gen of /$<x>=@(...)/ constructs.

An argument intended for INTERPOLATE was placed too high up in the tree. This not only meant it would not be passed, but confused the NFA builder, which then exploded (which is why compilation of the above construct failed). This fixes the code-gen, making things happy again.		 10:10
ast: 25168ae | jnthn++ | S05-capture/alias.t:
Test to cover RT #129249.
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129249
dalek kudo/nom: 135bba9 | (Zoffix Znet)++ | .travis.yml:
Ignore JVM failures

Rakudo's test suite has been failing for a while on JVM, but this went on silent on Travis because the test harness was erroneously indicating success with its exit code. The harness was fixed in 626a2220e7195 and now we're getting hammered on IRC with JVM test suit failures.		 10:11
Zoffix ^ I muted JVM failures for now. make test; needs to start passing again and right now it seems all of NativeCall tests fail right away with "dubious" result. 10:12
dalek kudo/nom: 63e6b81 | (Zoffix Znet)++ | t/04-nativecall/05-arrays.t:
Remove trailing whitespace
 10:43
kudo/nom: cc75c82 | (Zoffix Znet)++ | / (2 files):
Refactor dispatch

Since the slurpy just shuttles values to non-slurpy variant and can handle the cases for both multies, we do not need any multies at all.		 10:47
travis-ci Rakudo build errored. Jonathan Worthington 'Correct code-gen of /$<x>=@(...)/ constructs. 11:04
travis-ci.org/rakudo/rakudo/builds/159833750 github.com/rakudo/rakudo/compare/5...b044df1695
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
lizmat Zoffix: re 63e6b815aa , there is a reason for having 2 candidates 11:36
calling a method with @a with a signature of *@a is 6x slower than when if the signature is @a 11:37
travis-ci Rakudo build failed. Zoffix Znet 'Ignore JVM failures 11:38
travis-ci.org/rakudo/rakudo/builds/159833874 github.com/rakudo/rakudo/compare/c...5bba9a8643
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
lizmat Zoffix: whereas the overhead of shuttling from the *@a to the @a is only 16% extra, which the JIT will probably OSR anyway 11:39
when called sufficiently often enough
in fact: if this is all about handling the empty array / no parameters case, we probably want an extra candidate 11:42
dalek kudo/nom: 838c001 | lizmat++ | lib/NativeCall/Types.pm6:
Optimize CArray.new some more
 12:05
lizmat Zoffix: ^^^ feel free to revert if you think I missed the point :-)
lizmat goes back to translating 12:06
MetaZoffix lizmat++ thanks for fixing it. I'll keep in mind the cost of slurpies from now on. 12:08
travis-ci Rakudo build failed. Zoffix Znet 'Remove trailing whitespace' 12:27
travis-ci.org/rakudo/rakudo/builds/159839919 github.com/rakudo/rakudo/compare/1...e6b815aa66
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
MetaZoffix Hm. Why does it still report JVM failures? :/ 12:31
Weird. If I go to github.com/rakudo/rakudo/blob/nom/.travis.yml I see my changes in the file (... I think...) but my commit doesn't show up in history :/ 12:35
dalek kudo/nom: 4f0a551 | (Zoffix Znet)++ | .travis.yml:
Remove -os from allow_failures for JVM

To replicate the version in fcf27ae07eb411d0, as the current one doesn't appear to work to mute the failures.		 12:36
mst nine: oh good. now you see why I reacted so horrified at the original misunderstood version ;) 12:54
travis-ci Rakudo build failed. Zoffix Znet 'Refactor dispatch 13:04
travis-ci.org/rakudo/rakudo/builds/159840556 github.com/rakudo/rakudo/compare/6...75c825f90b
buggable [travis build above] ☠ Did not recognize some failures. Check results manually
dalek p: f960976 | jnthn++ | tools/build/MOAR_REVISION:
Bump MOAR_REVISION for EVAL leak fix.
 13:13
kudo/nom: 43b4f3d | jnthn++ | tools/build/NQP_REVISION:
Get MoarVM with EVAL leak fix.
 13:14
lizmat pulls, builds and spectests 13:17
jnthn That fixes RT #129161
synopsebot6 Link: rt.perl.org/rt3//Public/Bug/Displa...?id=129161
MetaZoffix Oh, sweet. jnthn++ :)
jnthn It's kinda hard to write a test. I pondered exposing memory use stats, but even then it's still kinda hard in the sense that VMs take a while finding their memory ceiling "sweet spot" 13:18
Anyway, the fix is in for now. moar-ha++ for helping find it. :-)
lizmat jnthn: perhaps test on a side-effect of the leak, like contexts hanging around when they shouldn't ?
jnthn Well, it'd be an interesting test then in so far as the best way to write it would be by taking a heap snapshot and looking at it. :-) 13:19
But yeah, that'd be the good way to do it :)
It's a bit of a yak shave though... 13:20
NeuralAnomaly: status 13:21
NeuralAnomaly jnthn, [✘] Next release will be in 2 days and 15 hours. Since last release, there are 49 new still-open tickets (0 unreviewed and 0 blockers) and 10 unreviewed commits. See perl6.fail/release/stats for details
jnthn 0 blockers. Yay :)
brrt \o/
MasterDuke_ lizmat: how expensive are where clauses in multis? 13:31
lizmat MasterDuke_: very, as they disallow optimisation
MasterDuke_ it struck me after looking at your recent optimization to CArray.new that the "if @values.elems" could be implemented as "multi method new(@values where *.elems) {...}" and "multi method new(@values) { nqp::create(self) }" 13:32
lizmat every call needs to check all the candidates, run the where condition code to see if there's a match
yeah, that would be expensive
MasterDuke_ i suspected that would be the case
lizmat anyway, that is just handling the case where you would do my @a; CArray.new(@a) aka, sending an empty array 13:33
the CArray.new() case is handled by the first candidate
MasterDuke_ yeah, i was just really thinking of that as an example (it triggered the thought) 13:34
the where clause needs to be run even if there's a tighter? (forgetting the precise word) multi? 13:35
lizmat MasterDuke: how can it be sure it is tighter without checking ? 13:36
MetaZoffix
.oO( narrower )
 13:37
MasterDuke_ well, i'd consider "multi nethod new() {...}" narrower than "multi method new(@values where {any code whatsoever})" 13:38
MetaZoffix++ that's the word i was looking for
lizmat MasterDuke_: true, that would work 13:39
MasterDuke_ obviously the implementation of checking candidates and such makes a difference, but at least it sort of seems in a high-level way, that the total amount of "logic" is the same 13:42
between "multi method new(@values) { if @values.elems {...} else { nqp::create(self) }" and "multi method new(@values where *.elems) {...}; multi method new(@values) { nqp::create(self) }" 13:43
MetaZoffix The one with .where would check .elems twice, because we use that value.
for $n
MasterDuke_ ooh, can you do "where *.elems -> $n"? because that would be cool 13:44
moritz yes
travis-ci Rakudo build failed. Elizabeth Mattijsen 'Optimize CArray.new some more'
travis-ci.org/rakudo/rakudo/builds/159854364 github.com/rakudo/rakudo/compare/c...8c0011acdf
buggable [travis build above] ☠ Did not recognize some failures. Check results manually 13:45
MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤0 pos␤|␤3 pos␤»
MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where say('CHECK!') && my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤CHECK!␤CHECK!␤0 pos␤|␤CHECK!␤CHECK!␤3 pos␤»
MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where .elems -> $n ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3] 13:46
camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> { say "slurpy" }; multi x(@ where .elems⏏ -> $n ) { say "$n pos" }; x; say '|'; x␤…»
MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where *.elems -> $n ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> say "slurpy" }; multi x(@ where *.elems⏏ -> $n ) { say "$n pos" }; x; say '|'; x␤…»
MetaZoffix moritz: yes?
Doesn't seem to work
moritz MetaZoffix: oh, in a signature 13:47
MetaZoffix: I missed that part, sorry
in mainline code it should work
m: where 42 -> $n { say $n }
camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> where 42⏏ -> $n { say $n }␤ expecting any of:␤ infix␤ infix s…» 13:48
moritz wait, what's the new if defined thingy called?
moritz totally out of the loop
MetaZoffix with
lizmat pretty sure the scope of a sub and the scope of the code executed in a .where are different
moritz m: whith 42 -> $n { say $n }
camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> whith 42⏏ -> $n { say $n }␤ expecting any of:␤ infix␤ infix s…»
lizmat *with
moritz m: with 42 -> $n { say $n }
camelia rakudo-moar 43b4f3: OUTPUT«42␤»
moritz helps to spell it correctly :-)
MetaZoffix lizmat: but the above code shows the scope is visible.
lizmat ah? 13:49
MetaZoffix And I know from IRC::Client that regex matches keep the $<captures>
m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where say('CHECK!') && my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤CHECK!␤CHECK!␤0 pos␤|␤CHECK!␤CHECK!␤3 pos␤»
MetaZoffix The "3 pos" is $n = .elems
lizmat hmmm... TIL :-)
moritz thunks don't introduce a new scope
m: sub f(@x where my $n = @x.elems) { say $n }; f <a b> 13:50
camelia rakudo-moar 43b4f3: OUTPUT«2␤»
moritz MetaZoffix: ^^ better? :-)
m: sub f(@x where my $n = @x.elems) { say $n }; f []
camelia rakudo-moar 43b4f3: OUTPUT«0␤»
moritz huh, why didn't that fail to dispatch?
lizmat because it's not a multi ?
jnthn Precedence
moritz m: sub f(@x where (my $n = @x.elems)) { say $n }; f [] 13:51
camelia rakudo-moar 43b4f3: OUTPUT«0␤»
jnthn m: sub f(@x where (my $n = @x.elems)) { dd @x }; f []
camelia rakudo-moar 43b4f3: OUTPUT«[]␤»
moritz lizmat: where should be checked for nonly's too, no?
jnthn Hmm
But I'm a bit surprised by that
moritz m: sub f(@x where (my $n := @x.elems)) { dd @x }; f []
camelia rakudo-moar 43b4f3: OUTPUT«[]␤»
MetaZoffix m: sub f(@x where (my $n = .elems) != 0) { dd @x }; f []
camelia rakudo-moar 43b4f3: OUTPUT«Constraint type check failed for parameter '@x'␤ in sub f at <tmp> line 1␤ in block <unit> at <tmp> line 1␤␤»
MetaZoffix It's using smatmatch 13:52
m: say so 0 ~~ []
camelia rakudo-moar 43b4f3: OUTPUT«False␤»
jnthn Oh, right
MetaZoffix m: say so [] ~~ 0
camelia rakudo-moar 43b4f3: OUTPUT«True␤»
moritz aaah
lizmat aaah TILA
jnthn m: sub f(@x where (so my $n = .elems) != 0) { dd @x }; f []
camelia rakudo-moar 43b4f3: OUTPUT«Constraint type check failed for parameter '@x'␤ in sub f at <tmp> line 1␤ in block <unit> at <tmp> line 1␤␤»
jnthn :) 13:53
moritz m: sub f(@x where so my $n := @x.elems) { say $n; dd @x }; f [42]; f []
camelia rakudo-moar 43b4f3: OUTPUT«1␤[42]␤Constraint type check failed for parameter '@x'␤ in sub f at <tmp> line 1␤ in block <unit> at <tmp> line 1␤␤»
lizmat something different: shouldn't this be a compile time error? 13:56
my @a[3]; @a[3] = "foo"
m: my @a[3]; @a[3] = "foo"
camelia rakudo-moar 43b4f3: OUTPUT«Index 3 for dimension 1 out of range (must be 0..2)␤ in block <unit> at <tmp> line 1␤␤»
lizmat I guess this is LHF in the optimizer :-)
MasterDuke_ moritz++ jnthn++ MetaZoffix++ lizmat++ (Perl 6)++ (where clause in multi-dispatch)++ 13:59
very cool
would it ever be possible to make it fast? 14:00
jnthn There's various ways we could make it faster 14:01
Caching a narrowed-down candidate list rather than doing the full slow-path dispatch would be one way.
MasterDuke_ that sounds sort of like a MoarVM thing? 14:04
jnthn I don't think it'd need MoarVM changes 14:05
Since MoarVM doesn't care exactly what's in the cache, it just invokes it
MasterDuke_ interesting, rakudo populates the cache? 14:06
jnthn So it should be OK if the multi-dispatcher code (written in NQP) decided to poke a closure in there
Yeah
MoarVM/JVM don't know Perl 6's multi-dispatch algorithm
(Which is nice, 'cus we only have to maintain it in one place :)) 14:07
MasterDuke_ where is the multi-dispatcher code? (mostly idle curiosity, but you new know...) 14:09
*never
jnthn BOOTSTRAP.nqp 14:10
(src/Perl6/Metamodel/) 14:11
MasterDuke_ thanks 14:16
travis-ci Rakudo build passed. Zoffix Znet 'Remove -os from allow_failures for JVM 14:24
travis-ci.org/rakudo/rakudo/builds/159860651 github.com/rakudo/rakudo/compare/8...0a551b8847
buggable [travis build above] ☠ Did not recognize some failures. Check results manually 14:25
MetaZoffix heh 14:26
Ah, right makes sense, 'cause JVM failures are still there
New blog post: "Perl 6 Core Hacking: Wrong Address; Return To Sender": perl6.party/post/Perl-6-Core-Hackin...-To-Sender 14:48
jnthn So, here is my rough thinking on improving our encodings handling: gist.github.com/jnthn/4b2ce730fe7b...57291de572 15:01
lizmat afk&
will read when back :-)
jnthn :) 15:02
Feedback welcome; will take a short break and bbi15 or so :) 15:03
travis-ci Rakudo build failed. Jonathan Worthington 'Get MoarVM with EVAL leak fix.' 15:08
travis-ci.org/rakudo/rakudo/builds/159870352 github.com/rakudo/rakudo/compare/4...b4f3d02faf
buggable [travis build above] ☠ Did not recognize some failures. Check results manually 15:09
MetaZoffix jnthn: the $incomplete in consume-line-chars() isn't ideal IMO... If I if I want to signal that no more bytes are coming, then I pass ":incomplete". But that can also be interpreted by person reading the code that the decoding is `incomplete` and there *are* more bytes coming. Maybe :no-more-bytes or :done or :end or something is a more clear name. 15:30
(FWIW, there's a typo in method name set-line-separaters ) 15:31
The last Travis failure is a real failure, but it looks like just a flopper in t/04-nativecall/13-union.t 15:32
jnthn MetaZoffix: Maybe :eof would be better 15:39
MetaZoffix sounds good
jnthn Tweaked that and the spello 15:41
MetaZoffix jnthn++ 15:43
jnthn Thanks for looking at it :) 15:44
nine Encoding.aliases really could provide a default implementation ;) 15:48
jnthn The empty list? :)
nine yes 15:49
jnthn Point. Changed.
MasterDuke_ minor, but "provide ore flexibility" 15:51
jnthn d'oh
Fixed it for the sake of other readers (don't expect that text to make it anywhere but this gist, though) 15:52
nine jnthn: so what about encoding/decoding errors?
jnthn Well, encoding should take a replacement
We already handle that, just need to work it into the API
Without that, it's a straight exception.
For decoding, it's always an exception at present. 15:53
(Which is a valid way to go per www.unicode.org/faq/utf_bom.html#gen8) 15:54
nine jnthn: metacpan.org/pod/Encode#Handling-Malformed-Data is probably very much inspired by real world use cases 15:55
jnthn Might be nice to also offer the replacement char functionality while we're doing this
nine I guess if I want to check if a bunch of bytes is decodable UTF-8 I just .decode and catch exceptions? 15:57
jnthn If you want to only decode valid UTF-8, then yes.
nine has had to deal with Maybe-UTF-8 far too often...
jnthn I think looking at that, we can reasonably provide the replacement char strategy too (and let the user pick that) 15:58
nine In Perl 5 I've written utf8::upgrade($content) unless utf8::decode($content) quite a few times 15:59
Though utf8::decode($value) unless utf8::is_utf8($value); is more common and I'm glad, we don't need it in Perl 6 :) 16:01
jnthn :) 16:03
timotimo returns from the autobahn 16:05
mst nine: "this appears to be encoded in WTF8"
jnthn nine: Added some stuff on replacements. 16:15
dogbert17 .seen MetaZoffix 16:17
yoleaux2 I saw MetaZoffix 15:43Z in #perl6-dev: <MetaZoffix> jnthn++
dogbert17 Zoffix: The spectest which fails consistently in Linux (Mint) 32 bit is t/spec/S32-io/socket-accept-and-working-threads.t 16:18
jnthn Odd...how does it fail? 16:19
dogbert17 jnthn: gist.github.com/dogbert17/b9e4a8e6...8b13c19ca3
timotimo that fails on my machine, too. it just consumes 100% of each of my 4 cores until i kill it 16:20
jnthn wtf
jnthn runs it in a loop for a while 16:21
Does it always fail in the same place for the two of you?
Just did 10 runs and it passed every time 16:22
dogbert17 I'm on 32 bit
jnthn I don't immediately see how that'd be a singificant difference.
dogbert17 If always fails a while after having printed 'ok 5 - Server responded (4)' 16:23
timotimo i'll fetch newest everything and give it a go
jnthn uh, wowzer 16:25
dogbert17 found something?
jnthn gist.github.com/jnthn/ddc28fba5043...626c4efaf4 16:26
See the maxresident on that! 16:27
If that's in KB then it really does try to allocate 2GB
dogbert17 I only have 1.5 gigs in my vm
just ran it in perl6-valgrind-m, big explosion :) 16:28
timotimo yeah, rakudo omnomnoms ram sometimes :)
jnthn 16GB in mine :)
Yes, but that's a rather insane amount :)
dogbert17 sometime I get 'can't allocate ... bytes' or the sjort message 'killed' 16:29
timotimo oh, it finished
dogbert17 if it actually uses ~2 gigs that would at least explain my failures 16:30
jnthn Killed is probably the OOM killer
timotimo 2139212maxresident 16:31
dogbert17 I could restart my vm with say 2.5 gigs an see what happens 16:32
timotimo perf says it spends by far the most time in multi_cache_find_callsite_args and multi_cache_add
b2gills could it be that gc doesn't happen the for loop? 16:33
timotimo if there's no gc safe point, that could happen
with valgrind on it seems like the server responds with '' instead of "don't hang up" 16:34
jnthn Yeah, noticed that 16:35
Hm, it does say no GC runs
timotimo hm. with a failed test "server responded (4)", will it ever ok the "started work was completed (4)"
jnthn In the profiler
timotimo: Depends how it failed 16:36
timotimo "got ''" is probably not enough info to figure out how exactly it failed?
jnthn No
timotimo in the profiler? isn't this async and thus uses multiple threads and as such the profiler output is likely useless? :)
i see 6% total time spent in run_gc 16:37
dogbert17 ran with perl6-valgrind-m, don't know if the out put is useful: gist.github.com/dogbert17/1c2820b5...9664718da7
jnthn timotimo: Yeah, I think the --profile output is bogus 16:38
Looking into --profile=heap now
Yeah, that has 5 snapshots
timotimo probably not useful, dogbert17
jnthn 1 which will have come at the end, so it did 4 in the normal course of execution
So it's not that it's never doing GC
timotimo first of all, if you don't add --full-cleanup to the arguments moar receives, you'll get a whole lot of stuff extra
but then the program was killed in the middle of doing business 16:39
dogbert17 yes it was, a sad state of affairs :(
jnthn > snapshot 3 16:40
Loading that snapshot. Carry on...
> summary
Total heap size: 40,090,806 bytes
That doesn't seem to acocunt for it too much :)
timotimo OK, so we have memory allocated that's not being counted in the reprs? 16:42
jnthn Something like
crap
Tried a run with massif and it was killed :/
Right near the end though 16:43
jnthn tries toning it down to 3 loop iterations
timotimo isn't getting anything sensible from adding a probe on malloc to perf 16:52
jnthn 99.18% (425,735,041B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc. 16:53
->73.45% (315,284,808B) 0x4FB37BB: MVM_fixed_size_alloc (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
| ->58.39% (250,639,520B) 0x4FE9C79: MVM_multi_cache_add (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
A huge volume of allocation comes from adding to the multi cache too?!
timotimo hah 16:54
clearly we're doing cache invalidation wrong
jnthn #define MVM_MULTICACHE_BIG_PROFILE 1 to see what it says
Multi cache for new reached 2048 entries 16:56
Multi cache for new reached 4096 entries
...
dogbert17 feels useless, should I RT in the meantime? 16:57
jnthn timotimo: Curiously, MVMMultiCache does implement unmanaged_size... 16:59
dalek Heuristic branch merge: pushed 81 commits to rakudo/better-test-pm6 by zoffixznet 17:10
Woodi jnthn: maybe encoding/decoding work can have some nice, userproof buffering subsystem ? probably part of streaming. also, if plan is to always read files :binary then user-implemented decoding can do eg. unpacking .jpeg into array of pixels. even if now it is just for text maybe one day it will be universal. also2: gist is about always returning Blob but serialization is similiar to encoding maybe it can 17:11
be do this way ?
cygx o/ 17:28
cygx throws gist.github.com/cygx/7d792e09b182a...82663dfddb into the ring again
timotimo Woodi: that doesn't make much sense. you're better off working with incoming Buf directly
cygx it didn't get any comments the last time I did :(
timotimo since jpeg (especially) isn't a stream-oriented format 17:29
Woodi: don't forget that if you want to do image loading in general, you have to handle many different bit depths, floating point vs integers, multiple layers, alpha-transparency, metadata, ... 17:30
jnthn timotimo: OK, think I found it. Turns out it's a confusion between flag indexes and arg buffer indexe 17:31
timotimo so it keeps adding the same thing over and over? 17:32
jnthn Yeah :)
'cus the arg index is truncated
timotimo heh.
jnthn Just needs 1 bit to fix :) 17:33
timotimo so every darn call ends up giving us a new node in the tree
jnthn yup!
timotimo does the tree get longer and longer and more and more lopsided?
jnthn Longer and longer...but the nodes are unreachable
And the reason it's such an incredible memory blow-up is that it's a concurrent data structure, so frees at safe points.
(Which come, but not before we've built thousands of copies of the cache) 17:34
timotimo ah, right
which reminds me of that other bug again where we had a tight loop of calls(?) and we ended up allocating loads of stack frames (at least i think it was that) and never reached a safe point in that tight loop 17:35
jnthn The test runs a bunch faster with that fixed too :P 17:40
cygx: Something along those lines looks like a reasonable direction. 17:41
Though I'd prefer a design without the runtime mixins... 17:42
Woodi btw. I remember some commit saying that "no allocations on variable access" (in some case). why access to some memory location needs to allocate things ? 17:43
jnthn Normally lazy allocation 17:46
dogbert17: Spectesting a fix at the moment
timotimo m: my @foo; my $a := @foo[10]; my $b := @foo[20]; my $c = @foo[30]; say @foo.perl; ($a, $b, $c) = (1, 2, 3); say @foo.perl 17:49
camelia rakudo-moar 43b4f3: OUTPUT«[]␤[Any, Any, Any, Any, Any, Any, Any, Any, Any, Any, 1, Any, Any, Any, Any, Any, Any, Any, Any, Any, 2]␤»
timotimo Woodi: here's an example where access will allocate
dogbert17 jnthn++, I'm very impressed 17:50
timotimo and we keep some lexical variables nulled out until they are first accessed 17:51
then a Scalar gets created, for example
dalek kudo/better-test-pm6: 9fcdaff | (Zoffix Znet)++ | lib/Test.pm6:
Add rough design notes
 17:53
p: 630fade | jnthn++ | tools/build/MOAR_REVISION:
Bump MOAR_REVISION for multi-cache fix.
kudo/nom: a2ae6cd | jnthn++ | tools/build/NQP_REVISION:
Bump NQP_REVISION for multi cache fix.
 17:54
jnthn dogbert17: That should do it; let me know if you run into any more issues with that test file.
dogbert17 jnthn: thx a bunch, I'll try it out asap 17:55
timotimo wow 17:56
that test is now so fast that its total time amounts to 4 times starting up 17:57
dalek kudo/nom: 0cf7128 | jnthn++ | docs/ChangeLog:
Add today's fixes to the ChangeLog.
 18:00
jnthn Enough for now :) 18:02
Woodi got it 18:37
dogbert17 jnthn: the test worked on my system as well. It looks as if memory usage is down as well :-) 18:38
timotimo i wonder when that bug impacted other workloads 18:42
not only was it causing a bunch of memory churn, but it also meant we had to go without a multi dispatch cache for anything with too many named parameters 18:43
jnthn timotimo: Quite possibly. I'm glad we found/fixed it, anyways. 19:32
travis-ci Rakudo build passed. Jonathan Worthington 'Bump NQP_REVISION for multi cache fix.' 19:36
travis-ci.org/rakudo/rakudo/builds/159951909 github.com/rakudo/rakudo/compare/4...ae6cd4744a
buggable [travis build above] ☠ Did not recognize some failures. Check results manually 19:37
jnthn Man, even when I succeed the bots think I failed :P 19:40
timotimo ;( 19:47
they are just jealous
dogbert17 which optimization level is the default for perl6? When starting perl6 I can set --optimize=num, but wat is the default? 20:12
timotimo i think for normal stuff it's 2, for the core setting it's 3 20:31
dogbert17 timotimo: thx 20:33
timotimo: btw, have you ever seen this, happens sometimes when running t/spec/S32-num/power.rakudo.moar. gist.github.com/dogbert17/ca96cc99...e71850f262 20:34
timotimo something didn't get sunk 20:37
normally, we rely on calls to .sink on results of stuff to be generated by code-gen to make failures throw, or otherwise handle them 20:38
so when it doesn't get its "handled" flag set to 1 before the GC kills it, we output a big warning
and that warning also contains the error "inside" the failure
it's interesting that the backtrace is inside the optimizer; it must have been compile-time evaluated? 20:39
dogbert17 it does not happen very often 20:40
timotimo right, it depends on the GC to be run at least twice between the Failure being created and the program ending 20:41
dogbert17 aha, so it's nothing to worry about then?
timotimo not 100% sure
it might be interesting to figure out which piece of code causes that failure to be generated 20:42
maybe the optimizer wants to learn about Failure objects when doing compile-time evaluation. perhaps it already knows to throw them away, but doesn't know to set them to "handled"
dogbert17 apart from that message I only have one test fail left, then 'make spectest' would run cleanly on my system 20:46
you and jnthn fixed the most serious failure, thx
the last test in t/spec/S32-num/power.rakudo.moar is the only one which still 'fails' 20:48
travis-ci Rakudo build failed. Jonathan Worthington 'Add today's fixes to the ChangeLog.'
travis-ci.org/rakudo/rakudo/builds/159953729 github.com/rakudo/rakudo/compare/a...f71280faa9
buggable [travis build above] ☠ Did not recognize some failures. Check results manually 20:49
Zoffix t/04-nativecall/13-union.t flop again 21:05
As for the previous... There *are* failures in JVM, which buggable sees, but travis ignores. So travis says passed
lizmat jnthn: re your encoding gist: 21:12
I would probably have written the encode method as:
method encode(Str:D $to-encode, *%options) returns Blob { 21:13
%options<replacement> = .NFC with %options<replacement>;
or is there a reason you want to have it as a separate named ? 21:14
jnthn lizmat: A little shorter, felt more natural...no strong reason. :) 21:16
Though .=NFC with %options<replacement>; probably works... :) 21:17
I was more interested in comments on the API tbh ;-) 21:18
timotimo: fwiw, we do detect Failure in the constant folder, but it's possible we fail to disarm them 21:19
lizmat need to sleep on that for a night, it's a lot to take in :-)
jnthn :-)
fwiw, the API of the streaming decoder isn't really "new"
It's largely a slight Perl 6-ification of the decode stream API inside of MoarVM that's been serving us well for a while now :) 21:20
(So in that sense it's a tested design.)
timotimo failure to disarm would be what causes this symptom i'd say 21:23
lizmat jnthn: slightly worried about performance 21:24
what are your thoughts about that ?
jnthn Performance of what in particular?
Note that for the currently supported encodings, the implementations of this API will delegate directly to the existing VM implemetation. 21:25
So really we're just pulling coordination of things up to Perl 6 level 21:26
Not hot lops
*loops
timotimo: If you're interested to investigate, then grepping for Failure in Optimizer.nqp will probably show up the appropriate point :) 21:27
timotimo ah, yes, right. 21:30
i could have a quick look 21:31
lizmat jnthn: ah, ok, yes :-) 21:32
timotimo yeah, easy peasy 21:33
lizmat sleep& 21:43
jnthn 'night 21:45
dalek kudo/nom: 7c48b90 | timotimo++ | src/Perl6/Optimizer.nqp:
prevent optimizer from causing unhandled failures in GC
timotimo dogbert17: ^- this commit fixes the spurious Failure objects in the GC with backtraces into the optimizer 21:52
actually, the fix wants to look a tiny bit different
dalek kudo/nom: 2c95f74 | timotimo++ | src/Perl6/Optimizer.nqp:
don't want to call .Bool on undefined Failure objects
dogbert17 timotimo: many thanks, man you're fast :-) 21:59
jnthn :) 22:00
timotimo would have been even faster if i had looked at it immediately, ond not waited for jnthn to poke me :)
jnthn timotimo++
timotimo jnthn++
dogbert17 jnthn++, timotimo++ 22:02
well, it's time for me to hit the sack, good night and thanks for the awesome help 22:06
timotimo gnite dogbert! 22:09
i might also go to bed soon 22:12
Zoffix I think I spotted a race condition when outputting to STDOUT and then immediately to STDERR: gist.github.com/zoffixznet/5bf9a6d...02c8875a06 22:15
That's our current Test.pm. Notice how the first run the two `not ok`s are together, but in the next run they're separated.
There's no async code in Test.pm. Is this 'cause we're using libuv's async out stuff? 22:16
Should I RT, or is this stuff well-known?
timotimo rosettacode.org/wiki/Chat_server#Perl_6 - the bug this refers to has been fixed and this task can now be fixed up 22:17
yeah, that's a race condition. you can only get them in the right order if you're quite fast i guess?
Zoffix OK :0 22:18
*:)
timotimo stackoverflow.com/questions/449781...chronously - shortest google search gives us this
Zoffix Doesn't matter much here, as the STDERR is just for humans
timotimo which i'm now reading
dalek kudo/better-test-pm6: 4fb849c | (Zoffix Znet)++ | / (5 files):
Implement ok/nok and all of the supporting infrastructure
 22:31
travis-ci Rakudo build passed. Timo Paulssen 'prevent optimizer from causing unhandled failures in GC' 23:19
travis-ci.org/rakudo/rakudo/builds/160011326 github.com/rakudo/rakudo/compare/0...48b90b013c
buggable [travis build above] ☠ Did not recognize some failures. Check results manually

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!