#perl6-dev on 4 April 2018 - Raku Programming Language Log

MasterDuke

m: for ^5_000 -> int $i { for ^5_000 -> int $j { } }; say now - INIT now

00:27

camelia

3.1955034

MasterDuke

m: for ^5_000 { for ^5_000 { } }; say now - INIT now

camelia

1.399760636

00:28

MasterDuke

huh, not what i was expecting

"Inlining eliminated the need to create 0 call frames (that's 0%)." -- native case

00:31

"Inlining eliminated the need to create 24989411 call frames (that's 99.92%)." -- non-native case

non-native version even spent 645ms doing 237 GCs vs 0 GCs for native version

00:33

nine

Zoffix: camelia JVM build fails with: Unhandled exception: VMArray representation does not implement bind_pos_native

07:22

in STORE (gen/jvm/CORE.setting:29581)

07:23

That's in github.com/rakudo/rakudo/blob/mast...y.pm6#L835

07:32

Zoffix: ooooooh... rakudo-jvm was still on nom on camelia! Building master now

07:39

timotimo

oh wow

08:01

nine: where you at? :)

nine

timotimo: not at GPW :( Too much work to afford GPW and PTS in April

08:14

timotimo

i see!

08:18

lizmat

:-(

08:22

nine: don't overwork! been there, done that, no fun :-(

08:23

|Tux|

drumroll …

08:32

timotimo rolls drums around

08:36

lizmat

ka-tjing! :-) ?

08:38

|Tux|

Rakudo version 2018.03-155-g5981f81b6 - MoarVM version 2018.03-76-g21d6c5470

csv-ip5xs	0.902	-	0.906
csv-ip5xs-20	8.933	-	9.246
csv-parser	36.379	-	36.903
csv-test-xs-20	0.471	-	0.482
test	8.335	-	9.080
test-t	2.382	-	2.396
test-t --race	1.005	-	1.010
test-t-20	42.715	-	42.866
test-t-20 --race	15.351	-	15.500

08:40

2018-04-02 12:13:31 test-t 2.400

2018-04-04 10:31:33 test-t 2.396

2018-04-04 10:36:12 test-t 2.382

2018-03-29 14:32:06 test-t 2.379

2018-03-29 14:36:32 test-t 2.372

second time two following tests under 2.4

08:41

lizmat

whee :-)

08:42

timotimo

fantastic

jnthn

.tell MasterDuke Because when the static optimizer understands all args are native it does the inline of the native operator then. So there's no inlining for spesh to do

09:45

yoleaux

3 Apr 2018 21:30Z <pmurias> jnthn: a big problem with the current parameter interning is that paramaterized types are registered when they are deserialized, and objects are deserialized lazily

jnthn: I'll pass your message to MasterDuke.

01:46Z <MasterDuke> jnthn: how come the inline log for 'my $s = 0; for ^3_000 -> $i { for ^3_000 -> $j { $s = $i - $j } }; say $s;' say `Can inline infix:<-> (3193) into (1)`, but not for 'my int $s = 0; for ^3_000 -> int $i { for ^3_000 -> int $j { $s = $i - $j } }; say $s;'?

pmurias

jnthn: do you have ideas for how the parameter interning should be fixed? I'm thinking maybe moving out the paramaterized registering to a separate section makes sense

10:58

jnthn

pmurias: Maybe by not lazily deserializing them.

12:04

Geth

rakudo: 7c51061134 | (Jonathan Worthington)++ | src/core/Rat.pm6
Avoid two Scalar allocations in various Rat ops

rakudo: 82b19b1f53 | (Jonathan Worthington)++ | src/Perl6/Actions.nqp
Use target coercion type in scalar wrap decision

When we compiler a signature with some `T $x`, if T is a type that is not `Iterable` then we don't need to wrap it in a `Scalar`. This saves quite some allocations. However, we used the nominal type always to decide whether to wrap. That's both creating containers when we don't need to (for example, `Bool()` doesn't need one but would get one), but also potentially a bug if one coerces to an iterable type from a non-iterable one (probably odd enough that we didn't hit this issue).

rakudo: 74b755992d | (Jonathan Worthington)++ | src/core/hash_slice.pm6
Use Bool() on postcircumfix:<{ }> adverbs params

This saves a `Scalar` allocation per time we do `%foo<bar>:exists` and similar. In programs doing a lot of such operations, those can add up.

lizmat

trivia question: how many Match objects does this code create: my $s = "a" x 100000; my @a = $s.match(/a/,:g)

13:15

answer: it creates 2 Match objects for each match :-

13:19

(

nine

but but but...why?

13:24

lizmat

I have no idea...

13:25

seems to me we have an optimization opportunity :-)

timotimo

i'll make a more precise measurement, gimme a sec.

the count is accurate

13:27

lizmat

other thing, inspired by corion's talk:

would this make sense? Channel.new.hyper

aka, using a Channel as a data source for batching, so that the code inside the hyper can send on the Channel

13:28

think recursive link extraction and URL-fetching

timotimo

yeah, once in !cursor_init and once in !cursor_start_cur

13:29

wrong

13:30

CURSOR_MORE seems to call cursor_start, though

via the user's code, though

13:33

so it's something our regex compiler emits?

|Tux|

Travis fails with a zef message (last lines on long log api.travis-ci.org/v3/job/359818489/log.txt )

14:09

anything I can do about that?

timotimo

looks like perl6 crashed?

14:12

Geth

nqp: 44d03f812e | pmurias++ | src/vm/js/Operations.nqp
[js] Make nqp::getattr in void use autovivify the attribute

14:20

nqp: f1d11804bd | pmurias++ | t/nqp/058-attrs.t
Test that nqp::getattr in void use autovivifies

dogbert17

[Tux] it's the file t/90_csv.t which manages to make Perl6 SEGV

14:47

|Tux|

reproducable.

14:48

damn, only once. I had coredumpsize limit 0. When I raised it, it didn't core anymore :(

14:50

dogbert17

I can probably reproduce it. As luck has it, there are plenty of core devs somewhere in Germany, drinking beer, and wating for something like this to fix :)

14:53

|Tux|, timotimo: gist.github.com/dogbert17/82210e04...14cb8a3fd6

14:56

seems to be test #164

14:57

|Tux|

dogbert17++

but why not always?

timotimo

oh, a low-level null pointer somehow makes it in there

14:59

dogbert17

perhaps spesh is involved

15:01

dogbert17 checks

timotimo: I suspect that it's something to do with inlining

15:03

|Tux| try running the test file like this: MVM_SPESH_NODELAY=1 perl6 t/90_csv.t

15:05

|Tux|

segfault in 171

15:06

dogbert17

yeah, my comment about test #164 was totally bogus

|Tux|

but still, not always. Most tests still pass

15:07

dogbert17

yeah, I'm pretty sure that the gang in Germany will figure this out

15:08

|Tux|: very good that you noticed this

15:09

|Tux|

\o/

dogbert17

releasable: next

releasable6

dogbert17, Next release in ≈17 days and ≈3 hours. 4 blockers. 0 out of 158 commits logged (⚠ 9 warnings)

dogbert17, Details: gist.github.com/5787bfa2825586e286...4e9336a0b9

dogbert17

timotimo, |Tux|: reported as github.com/MoarVM/MoarVM/issues/835

15:44

AlexDaniel

releasable6: uptime

releasable6

AlexDaniel, 20 hours, 42 minutes, and 30 seconds, 1511.957031MiB maxrss. This is Rakudo version 2018.03-134-g20495f097 built on MoarVM version 2018.03-56-g85fc758ce implementing Perl 6.c.

AlexDaniel

dogbert17: :'(

15:45

no way, not again…

dogbert17

AlexDaniel: what's up with the memory usage?

AlexDaniel

I did touch releasable6 recently, so could be my mistake somewhere…

but yesterday some other bot was leaking also

dogbert17

but not all of them?

15:46

AlexDaniel

shareable6: uptime

shareable6

AlexDaniel, 20 hours, 44 minutes, and 29 seconds, 511.511719MiB maxrss. This is Rakudo version 2018.03-134-g20495f097 built on MoarVM version 2018.03-56-g85fc758ce implementing Perl 6.c.

dogbert17

bisect: uptime

bisectable6

dogbert17, 4 hours, 42 minutes, and 2 seconds, 298.699219MiB maxrss. This is Rakudo version 2018.03-134-g20495f097 built on MoarVM version 2018.03-56-g85fc758ce implementing Perl 6.c.

AlexDaniel

that's still higher than normal I think?

dogbert17

looks higher

AlexDaniel

it's a recent-ish regression I think

15:47

but no idea how to bisect

dogbert17

but that's your area of expertise :)

AlexDaniel

not when bisectable itself is the problem xD

15:49

dogbert17

do you remember which version of Rakudo you had before updating to the SEGV free one you're running now?

15:51

AlexDaniel

dogbert17: no idea whatsoever.

15:59

filed it here (so that I won't forget): github.com/rakudo/rakudo/issues/1686

dogbert17: well, I think bots were running fine on 2018.03

16:00

so that's a good old=

R#1280 was closed 7 days ago, but I don't know if it was using rakudo HEAD at that moment

16:01

synopsebot

R#1280 [closed]: github.com/rakudo/rakudo/issues/1280 [regression] Memory “leak” in whateverable

AlexDaniel

20495f097 is actually 5 days old

anyway, I think I'll just have to experiment with it

16:03

probably there's a way to trigger it by asking the bots to do something

undersightable6: status

16:11

undersightable6

AlexDaniel, OK! Working on it…

AlexDaniel, 0 errors, 13 warnings: gist.github.com/859a12492bac861bbe...8192cad37d

16:13

AlexDaniel

weird…

yoleaux: ping

yoleaux

AlexDaniel: pong! (0 queued, 0 running)

AlexDaniel

buggable and ZofBot are down

others are up and undersightable6 is just acting stupid

16:14

thanks

dogbert17: fwiw you can (and maybe even should) submit tickets like M#835 directly (well… indirectly) against rakudo

16:16

synopsebot

M#835 [open]: github.com/MoarVM/MoarVM/issues/835 SEGV when running t/90_csv.t in

AlexDaniel

well, at least for me it's more convenient I guess, because blockers and other stuff only works there

16:18

and we have a stricter “don't close without tests” policy there

dogbert17++ for creating a ticket btw

16:19

dogbert17: by the way, what did you do to reproduce it?

16:20

how consistent is it?

dogbert17

AlexDaniel: for me it was very consistent if I ran it with MVM_SPESH_NODELAY=1

16:37

perhaps you can bisect it

16:38

also, should I close the bug and move it to Rakudo instead?

16:39

Zoffix

TFW: you find a solution to a bug, but it turns out you did it in a dream :()

16:51

Or rather, I did solve it in a dream but apparently MSVC doesn't provide long double :P

16:52

actually long double don't solve it, so yey :)

17:17

m: say 2.56/.2

19:38

camelia

12.8

Zoffix

dam :(

So it appears I managed to fix R#1651 but it costs 12.8x perf drop in nqp::div_In :(

synopsebot

R#1651 [open]: github.com/rakudo/rakudo/issues/1651 [math] Drift in Str -> Num -> Str conversion

Zoffix

I think I'm gonna commit, 'cause righter > faster. And then gonna give it another crack after CaR grant (if it's approved), since by then I imagine $Zoffix.numeric-knowledge will be a lot greater.

19:41

looks like the fix broke denormals -_-

19:49

tony-o

[Tux]: is Text::CSV broken ?

20:24

samcv

this was odd Unhandled exception: Probable version skew in pre-compiled 'gen/moar/CORE.setting' (cause: no object at index 140115913614632)

20:54

on starting a program. but then i restarted it and it didn't show up again..

20:55

seems i can trigger it again

gonna try a make clean and rebuild moar

20:56

oh may be rakudo then

nope. recompiling didn't fix it

21:03

recompiled moar, nqp and rakudo

MasterDuke

try removing a .precomp dirctory somewhere?

21:04

yoleaux

09:45Z <jnthn> MasterDuke: Because when the static optimizer understands all args are native it does the inline of the native operator then. So there's no inlining for spesh to do

samcv

there's no precomp in the folder i'm running the script from. though i could try a clean make of rakudo. not sure how much that does

21:06

oh this isn't good

21:07

num_objects is a MVMint32 but we cast it to a 64 bit int

could be totally unrelated though..

timotimo

m: say 140115913614632.msb

camelia

timotimo

wouldn't upcasting clear out the higher bits?

21:08

perhaps there's a problem with the format string of that error message on top of that

samcv

well we cast from int32 to int64

in MVM_sc_get_object where it throws. but that may not be the issue

or maybe it happens in another place idk

Zoffix

int32 to int64 isn't a problematic cast tho.

21:11

samcv

yeah. which is why it's probably not the cause

this is the file that the i see the issue on cry.nu/files/process.tar

21:12

i can't replicate with other scripts

if someone can tell me if running it ~10 times causes that error

timotimo

valgrind or asan are quiet?

samcv

valgrind at least is clean

21:13

i will check asan though

Zoffix

samcv: Stuck it in loop; works fine. On 2018.03-155-g5981f81b6 built on MoarVM version 2018.03-79-g95014f524

21:14

samcv

it doesn't get an exception if i compile it with ASAN

21:19

Zoffix

ZOFVM: Files=1292, Tests=153265, 144 wallclock secs (19.73 usr 3.04 sys + 3081.28 cusr 154.08 csys = 3258.13 CPU)

21:21

Either stresstest is totally awful as a perf eyeballing tool or my 12x slower div_In doesn't have much impact

21:22

MasterDuke

Unhandled exception: Probable version skew in pre-compiled 'gen/moar/CORE.setting' (cause: no object at index 140022413980824)

This is Rakudo version 2018.03-155-g5981f81b6 built on MoarVM version 2018.03-79-g95014f524

samcv

gist.github.com/776e6e9813a3403904...dc0f906afb here is a backtrace.

21:23

Zoffix: what platform are you on?

Zoffix

$ uname -a

21:25

Linux VirtualBox 4.2.0-30-generic #36~14.04.1-Ubuntu SMP Fri Feb 26 18:49:23 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

MasterDuke

samcv: what about an MVM_dump_backtrace?

samcv

do you want me to replace the throw with a backtrace instead?

ok this may be it. it starts as a MVMuint64 then it gets cast to an MVMint64. or could be unrelated. i'll at least test it

21:30

just tracing down the line of where that value comes from

21:31

MasterDuke

samcv: i meant `call MVM_dump_backtrace(tc)` where you also called `bt`

samcv

MasterDuke: call it inside gdb/lldb?

21:36

MasterDuke

yeah

samcv

MasterDuke: it doesn't print anything...

21:41

also prints nothing if i put the function above the exception throwing and recompile. and i know it runs since i put a printf in it

cur_frame is NULL

21:42

MasterDuke

put what function?

21:43

samcv

MVM_dump_backtrace

MasterDuke

oh, you don't need to put it in the source

21:44

just call it while stopped in gdb

21:45

samcv

well i tried it in lldb and it printed out nothing so i put it in the code to make sure. the issue is cur_frame is NULL

so it doesn't print anything out

MasterDuke

huh

21:49

samcv

yep

MasterDuke

is nine around? this seems like something he might know about

21:50

Zoffix

144 seconds was without my patch turns out

22:19

ZOFVM: Files=1292, Tests=153265, 149 wallclock secs (20.72 usr 3.14 sys + 3222.14 cusr 158.04 csys = 3404.04 CPU)

m: say 149/144

camelia

1.034722

Zoffix

ZOFVM: Files=1292, Tests=153265, 146 wallclock secs (20.50 usr 3.16 sys + 3114.82 cusr 157.74 csys = 3296.22 CPU)

22:23

Geth

nqp: a452de9a68 | (Zoffix Znet)++ | tools/build/MOAR_REVISION
[MoarVM Bump] Brings 11 commits

MoarVM bump brought: github.com/MoarVM/MoarVM/compare/2...6-gb735866 b735866 Fix drift in Num->Str roundtrip; Make nqp::div_In 14x slower 02dd490 Fix scanning loop in string cmp ASAN errors 9730ff9 jit continuationinvoke 19004b0 fix wrong pointer type ... (7 more lines)

22:40

¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...6-gb735866

rakudo: a760ac3cfc | (Zoffix Znet)++ | tools/build/NQP_REVISION
[NQP Bump] Brings 3 commits

NQP bump brought: github.com/perl6/nqp/compare/2018....2-ga452de9 a452de9 [MoarVM Bump] Brings 11 commits f1d1180 Test that nqp::getattr in void use autovivifies 44d03f8 [js] Make nqp::getattr in void use autovivify the attribute
... (12 more lines)

¦ rakudo: version bump brought these changes: github.com/perl6/nqp/compare/2018....2-ga452de9

roast: a03e959cfa | (Zoffix Znet)++ | S32-num/stress.t
Test Num.Str roundtrippage is stable

Covers R#1651 github.com/rakudo/rakudo/issues/1651

22:42

synopsebot

R#1651 [open]: github.com/rakudo/rakudo/issues/1651 [math] Drift in Str -> Num -> Str conversion

Geth

rakudo: fbab08163e | (Zoffix Znet)++ | t/spectest.data
Add S32-num/stress.t stresstest

Closes github.com/rakudo/rakudo/issues/1651 R#1651

22:43

roast: cd57ba0491 | (Zoffix Znet)++ | S02-names-vars/perl.t
Revert "Temp-fudge Num roundtrip test for Moar"

This reverts commit dba0ce16565f8003ca82e96c915998c08bb0e0f8.
The bug requiring this fudge has been fixed[^2];
[1] github.com/rakudo/rakudo/issues/1651
[2] github.com/MoarVM/MoarVM/commit/b7...30b335f27b

22:44

roast: 71f8f4ef9f | (Zoffix Znet)++ | S32-exceptions/misc.t
Revert "Loosen Num-in-sink warning text message match test"

This reverts commit 45468627033ef62abc4999268a6a6204fe0cf3f1.
The bug[^1] requiring this tweak has been fixed[^2];
[1] github.com/rakudo/rakudo/issues/1651
[2] github.com/MoarVM/MoarVM/commit/b7...30b335f27b

22:46

Zoffix

k; I think that covers all the Num-related stuff I wanted to do/fix before release.

22:47

timotimo

Zoffix: i wonder if you can make those shifts faster by doing direct bitwise operations on the underlying floating point numbers?

22:49

because 2^-1024 and 2^1024 ought to be just addition/subtraction to the exponent, right?

22:50

not even needing a normalization step of some kind?

Zoffix

No idea

22:51

$ ccc 'printf("%.17g", 2e0 << 100)'

22:52

program.c: In function ‘main’:

program.c:9:21: error: invalid operands to binary << (have ‘double’ and ‘int’)

timotimo

oh, i meant that differently

like, & it with 0b0111111111110000000000000000000000000000000000000000000000000000000

22:53

m: say "0111111111110000000000000000000000000000000000000000000000000000000".chars

22:54

camelia

timotimo

oops

Zoffix

I don't know if doubles are represented in the same format on all systems, with the exponent in that portion

Are they?

22:55

timotimo

that's standardised and i'm willing to bet every platform we're on uses that standard

how often does the shift happen?

22:56

m: say 0b11111111111

camelia

2047

Zoffix

once for most cases

timotimo

hm, i don't think it can actually do the shift more than once though?

or does that cause the fraction to become smaller instead?

22:57

Zoffix

I mean zero for most cases

timotimo

m: use NativeCall; my $num = 0.00000001e-1022; say nativecast($num, int64).base(16)

camelia

Native call expected return type with CPointer, CStruct, CArray, or VMArray representation, but got a P6int (int64)
in sub nativecast at /home/camelia/rakudo-m-inst-1/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 6…

timotimo

m: use NativeCall; my CArray[num] $num .= new(0.00000001e-1022); say nativecast($num, CArray[int64])[0].base(16)

22:58

camelia

timotimo

m: use NativeCall; my CArray[num] $num .= new(0.0001e-1022); say nativecast($num, CArray[int64])[0].base(16)

camelia

timotimo

m: use NativeCall; my CArray[num] $num .= new(1e-1022); say nativecast($num, CArray[int64])[0].base(16)

camelia

timotimo

is this the wrong way around perhaps? %)

Zoffix

What are you trying to do?

22:59

timotimo

it is

m: use NativeCall; my CArray[num] $num .= new(0.00000001e-1022); say nativecast(CArray[int64], $num)[0].base(16)

camelia

timotimo

look at the floating point number with its bits as hexadecimal

m: use NativeCall; my CArray[num] $num .= new(1e-1022); say nativecast(CArray[int64], $num)[0].base(16)

camelia

timotimo

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 1e-1022; say nativecast(CArray[int64], $num)[0].base(16)

camelia

Zoffix

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 1e-1020; say nativecast(CArray[int64], $num)[0].base(16)

23:00

camelia

Zoffix

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 1e-102; say nativecast(CArray[int64], $num)[0].base(16)

camelia

2AC1EAFF4A98553D

timotimo

i was literally getting zero? %)

23:01

Zoffix

Don't know why tho.

Oh doh

Yeah

timotimo

because base 2 vs base 10

Zoffix

I was looking at it as 2**-1020

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 5e-324; say nativecast(CArray[int64], $num)[0].base(16)

camelia

Zoffix

That's the lowest number

23:02

timotimo

now that we have firmly established that my brain is in no shape for thinking, i'll continue preparing my lightning talk!

Zoffix

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 1.7e-308; say nativecast(CArray[int64], $num)[0].base(16)

camelia

C396C98F8D898

Zoffix

that's close to highest number

m: use NativeCall; my CArray[num] $num .= new; $num[0] = 1.7976931348623157e+308; say nativecast(CArray[int64], $num)[0].base(16)

23:03

camelia

7FEFFFFFFFFFFFFF

Zoffix

That's DBL_MAX on my box

m: use NativeCall; my CArray[int64] $num .= new; $num[0] = 0xFFFFFFFFFFFFFFFF; say nativecast(CArray[num], $num)[0]

23:04

camelia

Cannot unbox 64 bit wide bigint into native integer
in method ASSIGN-POS at /home/camelia/rakudo-m-inst-1/share/perl6/sources/51E302443A2C8FF185ABC10CA1E5520EFEE885A1 (NativeCall::Types) line 109
in block <unit> at <tmp> line 1

Zoffix

m: use NativeCall; my CArray[int64] $num .= new; $num[0] = 0xFFFFFFFFFFFFFFFF-1; say nativecast(CArray[num], $num)[0]

camelia

timotimo

put 7 first instead of F that should work

Zoffix

m: use NativeCall; my CArray[int64] $num .= new; $num[0] = 0xFFFFFFFFFFFFFFFF-2; say nativecast(CArray[num], $num)[0]

camelia

Zoffix

m: use NativeCall; my CArray[int64] $num .= new; $num[0] = 0x7FFFFFFFFFFFFFFF; say nativecast(CArray[num], $num)[0]

camelia

NaN

timotimo

i think it complains before it even reaches the subtraction perhaps

right, the exponent being all 1s is for NaN

23:05

Zoffix

m: use NativeCall; my CArray[int64] $num .= new; $num[0] = 0x7FFFFFFFFFFFFFFF; say nativecast(CArray[num], $num)[0] xx 10, " batman"

camelia

(NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN) batman

timotimo

but anyway, if it doesn't call that often anyway, that's probably not the source of slowness

Zoffix

It's probably the division on huge integeres.

23:06

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!