travis-ci Rakudo build passed. usev6 'Avoid trailing semicolon for JVM backend 00:51
travis-ci.org/rakudo/rakudo/builds/273599064 github.com/rakudo/rakudo/compare/d...f8419a3e94
Rakudo build passed. Elizabeth Mattijsen 'Oops, need parens + comma inside nqp::stmts 03:12
travis-ci.org/rakudo/rakudo/builds/273600217 github.com/rakudo/rakudo/compare/1...62bcc49f25
Rakudo build passed. Timo Paulssen 'junction optimizer shall ignore proto and only look at candidates' 04:48
travis-ci.org/rakudo/rakudo/builds/273604714 github.com/rakudo/rakudo/compare/2...e858a555fe
lizmat Files=1223, Tests=67685, 292 wallclock secs (11.05 usr 4.67 sys + 1975.34 cusr 208.96 csys = 2200.02 CPU) 07:24
Geth rakudo/nom: 76f1d8970e | usev6++ | tools/build/Makefile-JVM.in
[jvm] Make libdir known in 'sub hll-config'

This lets EvalServer find Perl6::BOOTSTRAP and makes 'make test' work again. (At least after 'make install'.)		 08:26
[Tux] This is Rakudo version 2017.08-104-g76f1d8970 built on MoarVM version 2017.08.1-128-gde6dceda 09:19
csv-ip5xs 1.631 - 1.643
test 11.807 - 11.912
test-t 3.669 - 4.129
csv-parser 12.703 - 15.163
nine Oh...why so slow? 09:31
[Tux] don't know. will run a second time 09:37
This is Rakudo version 2017.08-104-g76f1d8970 built on MoarVM version 2017.08.1-128-gde6dceda 09:49
csv-ip5xs 1.332 - 1.339
test 9.799 - 9.869
test-t 3.472 - 3.600
csv-parser 10.526 - 10.918
[Tux] cannot explain
samcv i merged my collation stuff :) 10:18
sup [Tux]
[Tux] re-run again? 10:51
timotimo i wouldn't expect text::csv to use collation
[Tux] right 10:52
and the slowdown isn't as big in the second run
timotimo but i haven't actually looked to see what exact changes beyond that are in that branch. i think it's very self-contained though
oh yeah, i see it now
the slowest in the second run was faster than the fastest in the first
[Tux] running again, just for good measurement 10:53
This is Rakudo version 2017.08-104-g76f1d8970 built on MoarVM version 2017.08.1-128-gde6dceda 10:56
csv-ip5xs 1.360 - 1.362
test 9.595 - 9.890
test-t 3.508 - 3.536
csv-parser 10.751 - 10.818
have to go now. cu l8r
timotimo cu tux 10:58
Geth nqp: 731fcc381c | (Samantha McVey)++ | tools/build/MOAR_REVISION
Bump MoarVM: Unicode Collation Algorithm + other fixes

Changes: 2017.08.1-128-gde6dced..2017.08.1-150-g0b81969d 0b81969d Remove unneeded file from UCA implementation 866623d9 Merge Full Unicode Collation Algorithm Implementation 088aa0a0 Fix a leak in CArray repr. 0441e075 Ensure gi->start is set to 0 for flat strings in MVM_string_gi_init ... (13 more lines)		 11:02
Ā¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...-g0b81969d
rakudo/nom: 9b42484a5d | (Samantha McVey)++ | tools/build/NQP_REVISION
Bump MoarVM: Unicode Collation Algorithm + other fixes

NQP changes: 2017.08-47-g79e6453..2017.08-61-g731fcc381 731fcc381 Bump MoarVM: Unicode Collation Algorithm + other fixes 91d6538e0 Test passing wrongly typed required native named arguments 0e78e6b60 [jvm] When getting a wrong kind of native argument throw an exception instead of trying to coerce 8742805cb Add more index* tests to test empty string paths ... (28 more lines)		 11:03
Ā¦ rakudo/nom: version bump brought these changes: github.com/perl6/nqp/compare/2017....g731fcc381
MasterDuke rough charts of releases -> (time to parse SETTING, lines in SETTING): imgur.com/0uJW4U6
gist of the raw data is here: gist.github.com/MasterDuke17/215ad...290661fb96 11:04
timotimo, jnthn: ^^^
timotimo can you also make a "time per line" graph?
jnthn So the setting is at its biggest ever, but the parse time isn't quite at its biggest ever... 11:05
That's not too bad an outcome, though I am curious about the recent bump
MasterDuke imgur.com/JrtKN5E 11:11
jnthn Wow, I like that one more ;) 11:13
samcv MasterDuke, that's just the parse time?
jnthn Though still curious about the tick up at the end
MasterDuke samcv: yeah 11:14
unfortunately i haven't been able to profile the parse/build recently because it now requires more than 16gb of ram to --profile-compile the SETTING 11:15
oh, maybe i created a vm on google with their free credit that has enough ram, i should double-check that 11:16
timotimo yeah, --profile-compile is a heinous thing with such a big piece of code 11:27
the recursive nature of our parser, optimizer, and compiler forces the call graph to have roughly the same shape as the parse tree 11:28
MasterDuke i used to be able to do it on the bisectable server, but not anymore
jnthn Could probably do with a sampling profiler that just records stack traces at regular time intervals 11:29
To complement our current one 11:30
(Both are useful to have, but good at different things)
Skarsnik I ran out of memory again >< I hate this gumbo crash ;( 11:32
MasterDuke Skarsnik: how much memory do you have? Zoffix++ pointed me toward this free credit to use on a google vm, i think i have 24gb of ram configured for it 11:34
Skarsnik 4Gb, But I think it just don't crash running with efence, I can't get pass 12 iteration normaly and in gdb + efence it stopped at 35 because no memory left to mmap 11:35
timotimo jnthn: i entertained the thought to implement that recently, but i didn't have a good answer for how to interrupt the interpreter regularly for recording, and for how to make the jit also cooperate 11:36
Skarsnik that sucks lol 11:38
root@vps300582:/home/skarsnik# killall gdb
bash: fork: Cannot allocate memory
MasterDuke ! 11:43
perf report of parsing the SETTING: gist.github.com/MasterDuke17/ff994...366c9530a3 12:04
timotimo huh, rename_locals, eh? 12:07
ok, that's in spesh, so it's running off on its own thread anyway 12:08
MasterDuke i also added a report of doing the full build, not just parse 12:09
huh, MVM_string_equal at the top, don't remember seeing that happen before 12:11
Skarsnik that an interesting 'bug' I have 12:14
I get that in the calltrace : from /home/skarsnik/perl6/efence/rakudo/../../benchmark/perl6-gumbo/lib/Gumbo/Parser.pm6 but I did not use this path for this module. I put -I ../perl6-gumbo/lib wich is not in the benchmark dir 12:16
Ho because it use the one in the precomp dir of the xml module >< 12:19
timotimo ~i have a crazy idea, MasterDuke 12:24
MasterDuke i'm intrigued
timotimo i can set a breakpoint on MVM_string_equal, give gdb the commands "call MVM_dump_backtrace(tc)" and "c" and then output all that to a file 12:25
that file is going to become humongous, of course
i've never properly worked with mkfifo; i could probably make it output that to a fifo and have some compression program read from the fifo and write to disk? 12:26
MasterDuke yep, i've done that before (not for MVM_string_equa)
timotimo right, i thought i remember having done that once
do you remember what that was for?
MasterDuke or pipe stderr to gzip? 12:27
timotimo oh, right, it goes to stderr
MasterDuke yeah, you were the one who showed me how to do it. think i was recording where coerce_i was happeninig
timotimo oooh 12:28
yeah, we had too many nums in nqp
will you do the honors of recording all the MVM_string_equal?
i wonder how often it's called indirectly 12:29
i.e. not immediately from interp.c or from a jitted frame
MasterDuke i won't be able until late tonight, gonna be afk and traveling for the rest of the day
timotimo so maybe a "bt" would also be interesting
ah
safe travels, in that case!
i'll try my luck
MasterDuke i'll do it if i see you haven't already
thanks 12:30
timotimo cool, it segfaults when i try that :) 12:31
MasterDuke btw, slightly on-topic, any reason the id created by nqp::objectid couldn't be a string?
timotimo oh
dump_backtrace isn't using the gc correctly or something
hmm 12:32
i apparently reached a spot where there's no temproots in the TC yet 12:33
MasterDuke it's usually used as a string in rakudo, and it causes a bunch of coerce_I_s (or something like that), which means a lot of temporary allocations of mp_ints 12:34
i've never seen dump_backtrace segv before
timotimo yeah, tc->temproots was a null pointer 12:38
not sure how we managed that, and why it doesn't allocate one for us; perhaps weirdness with how functions called from gdb work
okay, now the thread context is a null pointer ... ?!? 12:40
MasterDuke --profile-compile of --target=parse for the SETTING: gist.github.com/MasterDuke17/7926d...0621e59b03 13:02
timotimo unspace is certainly taking a lot of time for not actually being used much or at all in the setting 13:08
it looks like some kind of compile-time setup for scheduler and awaitable is costing us significant time, too 13:10
it's literally just the role bodies being pointed out here 13:11
it's strange that these role bodies both have 590 entries 13:13
MasterDuke: can you find callers to these frames?
what the ... term:sym<name> in the list points at term:sym<nqp::const> for me 13:20
term:sym<name> is kind of complex, maybe we can find a fast-path or two to put in 13:21
oh, the lines don't match up because we actually put grammar and friends through a code gen step 13:26
Geth roast: 8d225fe907 | usev6++ | S06-macros/quasi-blocks.t
Use same ticket for skipped test
 13:54
AlexDaniel Skarsnik: you can try it on whateverable server 14:01
I'm afraid it only has 6 GB of ram out of 16 available once you start all bots and let them run for a whileā€¦ but I can put them all down for you :) 14:02
just let me know
Geth roast: 2fc07d161e | usev6++ | 2 files
[jvm] Unfuged passing tests
 14:05
Skarsnik AlexDaniel, not sure it worth it. I have the feeling it will not crash x) 14:34
AlexDaniel Skarsnik: ohā€¦ so what exactly should I do to make it stop crashing? ;) 14:37
Skarsnik I mean running in efence + gdb it crash before out of memory after 3x iterations instead of just 10-12 14:40
but I think there is something weird in github.com/supernovus/exemel/blob/...L/Node.pm6 at reparent/remove (called when you append a child on an xml::element) 14:43
Geth roast: b52bc57ff0 | usev6++ | 5 files
[jvm] Unfudge passing tests
 14:59
smls I've dug into Rakudo's sources to try and fix an RT, only to realize it probably *can't* be fixed without changing Capture semantics: 15:16
rt.perl.org/Ticket/Display.html?id=120995
Can a core dev maybe have a look and decide whether to reject the ticked? 15:17
*ticket
timotimo smls: don't forget we also compile signatures in many cases, so changing the code that handles this might not be enough, and in fact could cause "inconsistent bind result" errors 15:21
AlexDaniel smls: well, another option is to mark it as LTA and just leave it there. Who knows, maybe one dayā€¦ 15:28
smls AlexDaniel: But this way, we'll never get the number of open tickets down... :P 15:30
AlexDaniel smls: don't worry, there are 1500 other tickets to take care of :)
squashable6: status 15:31
squashable6 AlexDaniel, Next SQUASHathon in 25 days and ā‰ˆ18 hours (2017-10-07 UTC-12āŒUTC+14)
AlexDaniel I'm hoping that this ā†‘ will make a difference
Skarsnik: hey, have you tried building MoarVM with --asan ? 15:35
Skarsnik what dpoes that do? 15:37
timotimo it uses the address sanitizer lib to check memory accesses 15:39
AlexDaniel Skarsnik: files.progarm.org/2017-09-10-18394..._scrot.png
Skarsnik this look nice 15:40
AlexDaniel especially because it fails on the first iteration :)
Skarsnik I fixed this (efence did not like it etheir) 15:41
AlexDaniel timotimo: I guess I should've used --debug also to get pretty backtraces?
timotimo yeah 15:43
Skarsnik www.nyo.fr/~skarsnik/tmp/various-nc-stuff.diff the fix in nc_refresh make it stop crashing on this function
timotimo and maybe turn off the jit
that'll give better stack traces, too
Skarsnik If I understand correctly this try to get the addr on a field on a cstruct 15:44
but since it add a * too much it get the value I think
timotimo, do you think that could came from the XML module : github.com/supernovus/exemel/blob/...de.pm6#L46 I feel it make the gc happy sometime (I had a backtrace that point to this) 15:46
*does not make the gc happy 15:47
timotimo no clue what this does :)
i mean internally
Skarsnik it's called when I use append to add a child to an element 15:48
AlexDaniel, it's a configure option? 15:49
AlexDaniel yes
Skarsnik hm 15:50
root@vps300582:/home/skarsnik# apt-file search libasan.so
root@vps300582:/home/skarsnik#
timotimo your gcc doesn't come with it? 15:52
AlexDaniel it's in testing and unstable for sure 15:53
Skarsnik what the apt-file search give you?
timotimo i have a package "libasan" that i can install 15:55
in fedora
AlexDaniel a bunch of packages: libasan0 libasan1 libasan2 libasan3 libasan4 but also libgcc-5-dev 15:56
Skarsnik Oh it's with gcc 5
AlexDaniel and libgcc-7-dev also :)
Skarsnik: I applied your patch and now it's past 200 iterations without a crash 15:58
Skarsnik change the url to linuxfr.org
or something bigger like that ^^ 15:59
hm probably not available on my stable the libasan
AlexDaniel ok, that is much slowerā€¦ 16:00
and yeah, I see that it is eating more and more memory as it goes 16:01
Skarsnik apt-get -t testing install libasan4 work
but this replace lot of stuff x)
AlexDaniel Skarsnik: crashed on iteration 73 :| 16:03
Skarsnik well at least it crash x) 16:07
AlexDaniel now it crashed on iteration 74ā€¦ so it's not stable. Great
Skarsnik I am curious about something x) 16:08
Geth roast: d809d5651c | usev6++ | S02-names/pseudo.t
Add test for calling CORE::.keys (RT #129092)
 16:10
roast: 7edf26aae4 | usev6++ | S02-names/pseudo.t
[jvm] Unskip test for RT #123154
synopsebot6 Link: rt.perl.org/rt3/Public/Bug/Display...?id=129092
Link: rt.perl.org/rt3/Public/Bug/Display...?id=123154
roast: bae1108d8f | usev6++ | S32-hash/perl.t
Add test for RT #120656
synopsebot6 Link: rt.perl.org/rt3/Public/Bug/Display...?id=120656
AlexDaniel Skarsnik: but perhaps get your first fix in if it fixes something? 16:25
Skarsnik AlexDaniel, hm what you can try is saving to a xml file the first run to create xml file to load with the xml module and remove gumbo. to see it that came from xml or me not using the module properly 16:26
but the xml take forever to parse stuff xD
AlexDaniel, maybe, I am not sure. I guess I could do a PR and someone knowing more about this code could validate 16:29
timotimo, do you think it's a leak somewhere? www.nyo.fr/~skarsnik/tmp/leakxml.txt the code is like xml-load($somexml) for 1..100; and a call to getstatm each iteration 17:36
data = resident + stack If I remember correctly 17:37
timotimo looks like it plateaus reliably? 18:07
Skarsnik dunno it grows some time 18:09
still it's pretty crazy, 240Mb after the first iteration, then we reach 400Mb+ 18:10
timotimo yeah 18:11
have you considered the heap profiler? :) 18:12
Skarsnik I should give you the code. I am supposed to write a document xD 18:13
timotimo damn :) 18:14
it's dinner time here first
Skarsnik www.nyo.fr/~skarsnik/tmp/testxml.p6 the xml file is at the same url but any xml doc should do the trick x) 18:18
timotimo ah, that one does it without running, so no threads being created 18:21
Skarsnik timotimo> have you considered the heap profiler? :) I can picture you with a perl 6 book at my front door saying that 18:31
timotimo hah 18:55
lizmat timotimo: I'm looking deeper into converting buildplans to actual code 19:09
and am looking at github.com/rakudo/rakudo/blob/nom/...N.nqp#L108 19:10
and github.com/rakudo/rakudo/blob/nom/...N.nqp#L122
timotimo: it feels to me that every class has BUILDPLAN =:= BUILDALLPLAN 19:11
since we don't clone in L108 ?
hmmm.... scratch that
timotimo yeah, we go through the buildplan item by item 19:12
did you check out my old branch that tries to compile stuff
lizmat timotimo: no, first trying to get a grasp of things :-) 19:13
timotimo right
lizmat I wonder if we could same some memory for buildplans if mro[1] is Any, and just alias BUILDALLPLAN with BUILDPLAN ? 19:14
*save
timotimo dunno 19:15
i have a different idea how we could save memory for the buildplans
lizmat which is ?
change the structure of the data ?
timotimo instead of having lists of lists, we could have one native int array for the opcodes and then just lay all things flat into a single array and just shift as many pieces off of our iterator as every kind needs
yes
lizmat yes, I've been thinking along similar lines 19:16
timotimo cool, that means the idea isn't so bad :)
lizmat I'm also thinking that maybe we need to be smarter for the "has $.foo = 42" case
currently it calls a closure for initialization 19:17
if we could find out the closure generates a constant value, we could potentially simplify by just putting the value in the opcodes stream
timotimo ah, you mean if we know we have a literal we should just nqp::clone that
lizmat or even make it a "is default" value 19:18
timotimo ah, doing that ourselves
lizmat but yeah, don't call the closure
timotimo hm, how soon do we create that? probably too soon for the optimizer to have kicked in 19:22
lizmat yeah, before that 19:24
:-(
timotimo we should be able to figure something out still 19:25
and actually it's not too late in the optimizer to go through the buildplan and make it better, we just need to somehow remember the QAST::Block that was used for the is default 19:34
but maybe it'd be better if the actions themselves would figure things out for us 19:36
lizmat hmmm... so you're saying the optimizer should convert the buildplan to code ?
and then lose the buildplan?
hmmm... I guess we can't lose the buildplan because of mixins later, right? 19:37
timotimo hm, i'm not sure if the optimizer should be the one to do the code conversion
oh, right
we need to keep that, yeah
lizmat so basically we need a step that converts the buiildplan to code 19:38
possibly on demand ?
timotimo i don't really like running the compiler at run time 19:51
it'd be rather nicer if we did it at compile time and serialized the buildplans along with the objects 19:52
lizmat yeah, I agree on further thought 19:54
timotimo i find it bad enough that we start the compiler at run time when we use nativecall 19:57
Geth rakudo/nom: 7da0c2159e | (Elizabeth Mattijsen)++ | src/Perl6/Metamodel/BUILDPLAN.nqp
Don't copy BUILDALLPLAN if the same as BUILDPLAN

This should save some memory for every class that inherits from Any, or mixins into a class that don't add any attributes.		 20:19
timotimo should be visible in the CORE.setting.moarvm, too 20:22
lizmat timotimo: do you have any idea why some of the buildplan code uses @a[+@a] as an alternative to nqp::push ? 20:44
timotimo: also: sanity check: if a class does not have any attributes of its own, shouldn't we just set it to the buildplan of mro[1] ? 20:46
ah, no, we *could* have a BUILD or a TWEAK :) 20:47
afk& 20:48
timotimo good question, i'd imagine push ought to be faster. perhaps it used to be different waaaay back when? 21:00
MasterDuke timotimo, jnthn: i updated my recent gist of a --profile-compile ( gist.github.com/MasterDuke17/7926d...0621e59b03 ) to include the results from the last time i did it, which are very different. thoughts? 23:25
timotimo huh, the highest exclusive time is about 100x less now 23:27
yeah, i find this rather surprising
but i'll go to sleep now
MasterDuke yeah, but the total time hasn't changed much
timotimo is the sum of exclusive times still around the same? 23:28
MasterDuke hopefully inspiration will strike in your dreams
timotimo i hope so, too
i'll also need some inspiration for my grant application :)
seeya
MasterDuke later...
timotimo: 20170203 - sum(exclusive_time) = 52648660, max(inclusive_time) = 55981891 23:34
timotimo: 20170910 - sum(exclusive_time) = 77029801, max(inclusive_time) = 80924618 23:35
which is pretty much in line with the times i posted earlier 23:37

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!