| IRC logs at
Set by AlexDaniel on 12 June 2018.
01:03 Kaiepi joined 01:05 bingos_ joined, samcv_ joined 01:06 [Coke]_ joined 01:07 bonsaikitten joined 01:08 mst_ joined, ChanServ sets mode: +o mst_ 01:12 Kaiepi left, xiaomiao left, mst left, [Coke] left, BinGOs left, samcv left 01:37 [Coke]_ is now known as [Coke] 01:39 Kaiepi joined 02:46 vrurg left 03:23 vrurg joined 03:37 dogbert11 left, dogbert11 joined 04:43 dogbert17 joined 04:47 dogbert11 left 07:32 domidumont joined 07:41 Kaiepi left 07:49 brrt joined
nine MasterDuke: not yet. Have been quite busy with work and aircraft maintenance 07:58
japhb Aircraft maintenance? You own your own? 08:00
08:09 bingos_ is now known as BinGOs
nine No, they're owned by the club. I helped with maintenance of and am the primary maintainenance engineer of 08:09
08:09 BinGOs left, BinGOs joined
japhb That is very cool. Awesome choice of hobby and/or second job. :-) 08:11
nwc10 but this is in Linz? So how come they are German registered? 08:27
nine japhb: it's funny that I started doing maintenance with pretty much no prior experience, but I could take a lot of what I learned with aircraft to maintaining my bicycle and stuff in the house hold. I'm pretty sure it's usually the other way round ;0 08:28
08:28 notagoodidea joined
japhb I mean, if the Wright Brothers mythology is to be believed at all, yeah. ;-) 08:29
nine nwc10: it used to be that registering in Germany was simply cheaper and thanks to the EU it doesn't matter where it's registered. Nowadays we register in Austria as the difference disappeared
notagoodidea MasterDuke: For what is worth, I kept digging in rakudo to optmize a bit after you hint me to AT-KEY and I landed on this version : 08:31
For "reasons" applying .lc to lines instead of the inner for loop is faster (but seems less stable in performance with pikes doubling the time) and using is default(0) trait to do a Int comparaison seems faster than using .EXISTS-KEY. 08:33
The simple version was around ~22s on my computer to ~9s now. 08:39
08:44 Kaiepi joined
MasterDuke notagoodidea: is kind of cheating, because it's concatenating all the lines together and then calling lc on that. so you aren't really processing line-by-line anymore 08:46
notagoodidea Oh, that's why. 08:50
The time penalty is heavy to move back lc to the inner loop. 08:52
MasterDuke i have a possible patch for rakudo that slightly speeds up the BagHash version, will PR that for comments in a bit
yeah, it's ~1m calls to .lc instead of just one 08:53
i don't know if we're allowed to assume ascii in the optimized version, but opening/reading with :enc<ascii> would be faster 08:55
08:56 sena_kun joined
notagoodidea Yep, I was looking how to skip the NFC because we can assume ascii by the rules but on $*IN it doesn't seem possible? 08:56
09:12 frost-lab joined 09:21 zakharyas joined 09:51 brrt left
jnthn ASCII may be in NFC but it might not be in NFG because \r\n is a single grapheme 10:11
MasterDuke i think `$*IN.encoding("ascii");` will do it
jnthn (But yes, the ASCII decoder can avoid most of the work.) 10:13
notagoodidea I was doing open($*IN, :enc<ascii>).lines(:close).
MasterDuke fwiw, it doesn't seem faster for me for this example 10:14
notagoodidea hum
The open method is a bit faster because it just remove one level of indirection (I took it from the lines method in the IO/Handle source) 10:16
I think. But $*IN.encoding("ascii") set up something, it just return ascii or it must be use with the call to IO.CatHandle etc? 10:17
10:19 brrt joined
jnthn It changes the currently open handle so that reads from it in the future will use the ASCII decoder 10:25
10:27 Kaiepi left
MasterDuke interesting. adding `$*IN.encoding("ascii");` doesn't really make it any faster. however, if i do `for open("kjvbible_x10.txt", :enc<ascii>).lines` vs `for open("kjvbible_x10.txt").lines` it is about 1s faster (although `for open.lines` is itself about 1s slower than `for "kjvbible_x10.txt".IO.lines` or `for $*IN.lines` 10:42
10:44 brrt left
notagoodidea I think it is due to .lines being implementend as open(..).lines() for IO::Handle 10:53
And on the other side, use codesections response on SO ( with a custom reverse for sort (without dropping to nqp) cut the sort time by half. 10:55
MasterDuke are you using a release or building rakudo from source? 10:56
notagoodidea The rakudo-pkg release (2020.02 I think) 10:57
(On fedora 33 if it can matters)
MasterDuke ah, i'm building from source so i have the recent commits that (at least mostly) fix that 10:58
notagoodidea (2021.02 sorry)
It is also possible to shove some times if the .put for %w form could be avoided to a way to print without for loop put respecting the constraints of 1 pair by line. 11:00
nine jnthn: nice riddle you crated there :D
MasterDuke jnthn: a perf report of this example shows the most expensive function is MVM_multi_cache_find_callsite_args. i assume this oddity can be ignored because new-disp is likely to change things? 11:01
nine nqp::until(nqp::defined($continuation), nqp::null), i.e. busy waiting would also fix it. I wonder if that'd be a better solution? The situation should be pretty rare and I guess the busy wait would be much cheaper than $l.protect 11:05
jnthn MasterDuke: At the very least it's not going to exist after new-disp 11:13
nine: Hmmmmmm. 11:14
MasterDuke cool 11:19
nwc10 jnthn: what order do you expect new-disp and RakuAST to "land" in? 11:20
jnthn new-disp first
By quite a long way
Doing that way around means less re-work of things in RakuAST 11:22
It can just depend on the new-disp way from the start 11:23
MasterDuke hm. the BagHash version of this example can be sped up by changing to replace the existskey with the atkey 11:25
but that's only faster if the exitskey is usually true 11:26
lizmat hmmm... interesting point! 11:29
MasterDuke: what are you using as benchmark ? 11:30
MasterDuke `my BagHash $w .= new; $w.add($ for "kjvbible_x10.txt".IO.lines; say .key, .value for $w.pairs.sort(-*.value);` 11:31
i think the `my %w := bag "kjvbible_x10.txt".IO.lines>>.lc.words; say .key, .value for %w.pairs.sort(-*.value);` version also had ADD-ITERATOR-TO-BAG as the most expensive function 11:36
now this is interesting. i did see an improvement in that example with the change made. but an nqp micro-benchmark doesn't 11:42
nqp: my %a; %a<b> := 1; my int $c := 0; my int $i := 0; my num $s := nqp::time_n(); while $i++ < 100_000_000 { if nqp::existskey(%a, "c") { my $b := nqp::atkey(%a, "c"); $c := $c + $b }; }; say(nqp::sub_n(nqp::time_n(), $s)); say($c) 11:43
camelia 0.6337792873382568
MasterDuke nqp: my %a; %a<b> := 1; my int $c := 0; my int $i := 0; my num $s := nqp::time_n(); while $i++ < 100_000_000 { if (my $b := nqp::atkey(%a, "c")) { $c := $c + $b }; }; say(nqp::sub_n(nqp::time_n(), $s)); say($c)
camelia 1.081580400466919
MasterDuke the key isn't found and existskey is faster, no surprise
nqp: my %a; %a<c> := 1; my int $c := 0; my int $i := 0; my num $s := nqp::time_n(); while $i++ < 100_000_000 { if nqp::existskey(%a, "c") { my $b := nqp::atkey(%a, "c"); $c := $c + $b }; }; say(nqp::sub_n(nqp::time_n(), $s)); say($c) 11:44
camelia 2.2597131729125977
MasterDuke nqp: my %a; %a<c> := 1; my int $c := 0; my int $i := 0; my num $s := nqp::time_n(); while $i++ < 100_000_000 { if (my $b := nqp::atkey(%a, "c")) { $c := $c + $b }; }; say(nqp::sub_n(nqp::time_n(), $s)); say($c)
camelia 2.331805944442749
MasterDuke the key is found and existskey is still faster, surprise
and i actually see a bigger difference locally in the case where the key is there. 2.2s for existskey and 2.6s for atkey 11:46
oh, and that was on my remove_spesh_optimizations moarvm branch. on master the atkey version is slower still, 2.9s 11:48
11:51 Kaiepi joined 12:08 zakharyas left
lizmat MasterDuke: preliminary tests show that maybe 1-2% can be gained on BagHash.add in the case of keys already existing, by using nqp::ifnull(nqp::atkey 12:13
so that a lookup would only need to be done once 12:14
the same opt could be done for basically all of the methods in src/core.c/Rakudo/QuantHash.pm6
what I *did* find is that 20% of CPU is used by prefix<--> in the case of my %bh is BagHash; %bh.add(42 xx 1000000 12:16
so I'm going to focus on that 12:19
12:19 Geth joined
lizmat MasterDuke: 12:19
on master, this segfaults most of the time: 12:31
$ raku --profile -e 'my @a = 42 xx 10_000_00'
12:48 sena_kun left 12:52 sena_kun joined
MasterDuke so far it hasn't segfaulted for me even with 100_000_000 12:57
12:57 MasterDuke left, MasterDuke joined 14:07 zakharyas joined
nine Huh? unless nqp::defined($continuation) { $l.lock; $l.unlock; } nqp::continuationinvoke... does not actually fix the issue. Taking the lock undiscriminately does. As does busy waiting and ignoring the lock. 14:45
14:47 frost-lab left
nine Ah, it should be unless nqp::defined(nqp::decont($continuation)) 15:55
or no...that's not it either
jnthn nqp::isconcrete is perhaps a safer bet 16:05
But still not sure why it'd be wrong
nine What's also odd is that I just cannot provoke the error without heavy system load, even if I add a huge delay before the $continuation := c; 16:06
But as soon as I run some TEST_JOBS=80 make stresstest, it fails 16:07
afk for an hour 16:08
16:10 evalable6 left 16:12 evalable6 joined
nine Finally....s/huge delay/gigantic delay/ did the job 17:20
17:37 zakharyas left 18:19 notagoodidea left 18:24 zakharyas joined 18:29 domidumont left 18:56 lizmat_ joined 19:00 lizmat left 19:18 zakharyas left 19:34 brrt joined 20:30 brrt left 21:28 lizmat_ is now known as lizmat 21:54 mst_ is now known as mst 23:32 dogbert17 left 23:33 dogbert17 joined 23:48 dogbert17 left 23:49 dogbert17 joined 23:51 dogbert11 joined 23:54 dogbert17 left