| timotimo | i have a workload that spends about 90% of run time in collapse_strands | 00:03 | |
| samcv | wow it prints way too much with llvm 5.0 if i do by function. everywhere everything is inlined which may not be the best for the overall summary | 00:04 | |
| hmm timotimo | |||
| timotimo, is that workload just normal use of JSON::Fast? | 00:55 | ||
| timotimo | hm | ||
| well, it takes a ~420mb file of strings separated with null bytes | 00:56 | ||
| then it splits that into an array | |||
| that array gets serialized with to-json, then immediately deserialized with from-json | |||
| and then i join it with \0 to spurt it back into a different file | |||
| to see if it successfully roundtrips | |||
| anyway, it's bedtime for me now, i'd say | 00:58 | ||
| samcv | night night timotimo :) | 01:01 | |
|
01:28
MasterDuke joined
01:48
ilbot3 joined
02:15
quotable6 joined
02:50
agentzh joined
|
|||
| samcv | .seen brrt | 03:24 | |
| yoleaux2 | I saw brrt 4 Apr 2017 09:16Z in #moarvm: <brrt> the jit logging needs to be cleaned up anyway | ||
|
03:47
agentzh joined
04:37
spebern joined
05:22
agentzh_ joined
05:28
spebern left
05:39
geekosaur joined
05:45
domidumont joined
05:48
domidumont joined
06:06
brrt joined
|
|||
| Geth | MoarVM: 33af46191a | (Samantha McVey)++ | src/strings/normalize.c Remove two funct. in normalize.c that have been superseded is_spacing_mark and is_regional_indicator have been superceeded by the switches for the Grapheme_Cluster_Break property in should_break() which is also in normalize.c Was re-reminded about their non-use while examining our new coverage reports |
06:12 | |
| samcv | hello brrt | ||
| brrt | hi samcv | ||
| samcv | have you seen the new even fancier coverage | ||
| brrt | not yet, i think | 06:13 | |
| samcv | cry.nu/coverage/ for nqp and cry.nu/coverage/roast for roast | ||
| and you can link to line numbers in the files as well | |||
| and has a link at the top to 'jump to the first uncovered line' in a file | 06:14 | ||
| link line numbers like github | |||
|
06:16
domidumont joined
|
|||
| Geth | MoarVM/even-moar-jit: fd0de679cc | (Bart Wiegmans)++ | docs/jit/plan.org Document plan for 'let' to 'do' conversion This way, we can remove the 'internal rooting' mechanism and avoid a number of potential errors, as well as ensure that the template code as written is compiled as the developer should expect it, i.e. in linear, greedy order. |
06:24 | |
|
06:38
domidumont joined
06:44
domidumont joined
06:48
brrt joined
06:56
agentzh joined
|
|||
| samcv | brrt, let me know if you want me to profile roast and/or nqp test suite with the even-moar-jit branch | 07:44 | |
| brrt | hmmm | 07:47 | |
| that is actually a pretty good idea | 07:48 | ||
| samcv | yeah? | ||
| brrt | i think so, yes | ||
| samcv | ok roast or nqp test suite. which one should i do | ||
| brrt | nqp test suite will do fine | ||
| samcv | ok cool | ||
| brrt | it's pretty limited for now, after all | ||
| samcv | oo can you merge master back in on it first tho | 07:49 | |
| cause it doesn't have the new ./Configure.pl option | |||
| hmm got a merge problem. i'll just cherry pick it temporarly. will be faster | 07:56 | ||
| brrt | merge problem? | 07:58 | |
| i can merge it again, yes | |||
| the problem is that some of the Configure.pl options are WRONG | 07:59 | ||
| especially the ones regarding dynasm and lua | |||
| can't risk that, we need our own lua | |||
|
08:02
zakharyas joined
|
|||
| samcv | ok started generating it, will be done in 5 mins or so | 08:02 | |
| brrt | cool | 08:03 | |
| samcv | uh oh. nah. you should merge | 08:04 | |
| heh. it's missing indexic_s op too | |||
| and nqp is unhappy | |||
| well. it can't compile :P | |||
| brrt | alright, had no idea you were going so fast :-P | 08:06 | |
| re: rust evangelism, the one thing that made me chuckle was 'in the future when overflow checking is going to be cheap' | 09:29 | ||
| samcv | when will that be lol | 09:31 | |
| or is that the joke | |||
| brrt | okay, trivia question | 09:34 | |
| do you know why you do get a 'divide by zero' error and not an integer overflow error? | |||
| (well, you would, on some languages, but in C for example) | 09:35 | ||
| samcv | you mean compilier error? | 09:36 | |
| brrt | no, rutnime | ||
| (anyway, lunch &, will be back in a bit and leave you in suspsense) | |||
| samcv | kk | ||
| but i mean it doesn't make much sense to divide by zero anyway and i doubt it'd be compatible with the division algorithms they have | 09:38 | ||
| brrt | i have forgotten how the binary division algorithm typically works | 10:07 | |
| anyway, the reason is that a divide-by-zero raises a (hardware) interrupt, whereas integer overflow sets a bit in the flags register | 10:08 | ||
| so the one automatically draws you away from normal execution; the second you have to check-and-branch | 10:22 | ||
| and you can do this, but it doubles the size of your numeric code | |||
| rather than add %rax, %rdi; you have add %rax, %rsi; jo label;' everywhere | 10:23 | ||
| and if you want resumeability, you have to translate that into a call | |||
|
11:53
AlexDaniel joined
12:30
synopsebot6 joined
12:57
AlexDaniel joined
13:11
domidumont joined
15:08
brrt joined
|
|||
| Geth | MoarVM/even-moar-jit: 364e0d2ba1 | (Bart Wiegmans)++ | 7 files Extract expr op reading into module To make reading expr ops simpler and consistent, extract the expr ops into their own file, and create a module for reading it. |
15:31 | |
|
16:33
agentzh joined
16:49
agentzh joined
17:00
agentzh joined
17:12
agentzh joined
17:29
domidumont joined
18:31
AlexDaniel joined
19:08
praisethemoon joined
|
|||
| samcv | good * everyone | 19:35 | |
| lizmat | samcv o/ | 19:47 | |
| jnthn: what would you think of having a hash-like equivalent to IterationBuffer such as IterationHash (with the same REPR as nqp::hash, but without the auto-HLLizing everywhere) | 19:48 | ||
| s/IterationHash/IterationSer/ | 19:49 | ||
| *Set | |||
| grrrr ;) | |||
| timotimo | HashBuffer? LLHash? | ||
| lizmat | the thing is that it's very hard to make hotpath subs that take an nqp::hash as a parameter | 19:50 | |
| case in point: Set.new-from-pairs and Iterable (|) Iterable | 19:51 | ||
| I would like to apply the logic of new-from-pairs in the same nqp::hash | |||
| but if I put this in a sub, I need to de-HLLize everything again and again | 19:52 | ||
| samcv | nqp::hash's get HLLized? | 19:53 | |
| lizmat, does that have any relation to what happens when i assign a nqp::hash to a %sigil | 19:54 | ||
| m: my %hash = nqp::hash; %hash<foo> = 'bar'; say %hash | |||
| camelia | ===SORRY!=== Error while compiling <tmp> Could not find nqp::hash, did you forget 'use nqp;' ? at <tmp>:1 ------> my %hash = nqp::hashā; %hash<foo> = 'bar'; say %hash expecting any of: argument list |
||
| samcv | m: use nqp; my %hash = nqp::hash; %hash<foo> = 'bar'; say %hash | ||
| camelia | {foo => bar} | ||
| samcv | also what does HLL stand for ;) i have not dived into that section of the codebase yet | 19:55 | |
| lizmat | samcv: the (empty) nqp::hash gets HighLevelLanguaged, is empty, so doesn't assign anything to the %hash | ||
| samcv | it does though | 19:56 | |
| lizmat | m: use nqp; my $h := nqp::hash("a",42); sub a(\h) { say h.WHAT }; a $h | ||
| camelia | (Hash) | ||
| samcv | it assigns something to the %hash | ||
| lizmat | m: use nqp; my %h = nqp::hash; dd %h | ||
| camelia | Hash %h = {} | ||
| lizmat | samcv: don't think so | 19:57 | |
| samcv | m: use nqp; my %hash = nqp::hash; say nqp::existskey(%hash, 'blah') | ||
| camelia | 0 | ||
| samcv | m: use nqp; my %hash; say nqp::existskey(%hash, 'blah') | ||
| camelia | (signal SEGV) | ||
| samcv | see | ||
| it does *something* at least | |||
| lizmat | what does something? | ||
| samcv | i can't do nqp::existskey on a %sigil that is unpopulated or else it segV's | 19:58 | |
| but if i assign nqp::hash to the %sigil thing, then it doesn't segv and nqp::existskey works | |||
| lizmat | m: my %h; dd nqp::getattr(%h,Map,'$!storage') | 19:59 | |
| camelia | ===SORRY!=== Error while compiling <tmp> Could not find nqp::getattr, did you forget 'use nqp;' ? at <tmp>:1 ------> %h; dd nqp::getattr(%h,Map,'$!storage')ā<EOL> |
||
| lizmat | m: use nqp; my %h; dd nqp::getattr(%h,Map,'$!storage') | ||
| camelia | Mu $!storage = Mu | ||
| lizmat | m: use nqp; my %h = (); dd nqp::getattr(%h,Map,'$!storage') | ||
| camelia | {} | ||
| lizmat | if you don't try to assign to a Hash, it will not allocate its internal nqp::hash | ||
| samcv | m: use nqp; my %hash= (); say nqp::existskey(%hash, 'blah') | 20:00 | |
| camelia | 0 | ||
| lizmat | m: use nqp::existskey(My,'foo') | ||
| camelia | ===SORRY!=== Could not find nqp::existskey at line 1 in: /home/camelia/.perl6 /home/camelia/rakudo-m-inst-2/share/perl6/site /home/camelia/rakudo-m-inst-2/share/perl6/vendor /home/camelia/rakudo-m-inst-2/share/perl6 CompUnā¦ |
||
| lizmat | m: use nqp; nqp::existskey(My,'foo') | ||
| camelia | ===SORRY!=== Error while compiling <tmp> Undeclared name: My used at line 1 |
||
| lizmat | m: use nqp; nqp::existskey(Mu,'foo') | ||
| camelia | This type (Mu) does not support associative operations in block <unit> at <tmp> line 1 |
||
| samcv | kk. so it's not specific to assigning to nqp::hash. just that it has to have a hash allocated to it | ||
| lizmat | m: use nqp; nqp::existskey(nqp::null,'foo') | ||
| camelia | This representation (Null) does not support associative access (for type VMNull) in block <unit> at <tmp> line 1 |
||
| samcv | and the hash that gets allocated isn't the one i assign to it | ||
| lizmat | m: use nqp; my %h = nqp::hash # nqp::hash becomes {}, which becomes (), which causes $!storage to be allocated, but still empty | 20:01 | |
| camelia | ( no output ) | ||
| lizmat | samcv: ^^ what happens if you say %h = nqp::hash() | 20:02 | |
| samcv | kk | ||
| yeah. so something happens but not directly | 20:03 | ||
| lizmat | m: use nqp; my $h := nqp::hash; nqp::bindattr(my %hash,Map,'$!storage',$h); nqp::bindkey($h,"a",42); dd %hash | ||
| camelia | Hash %hash = {:a(42)} | ||
| lizmat | samcv: have a low level hash be part of a HLL Hash | 20:04 | |
| *having | |||
| samcv | hmm MVM_STRING_GRAPHEME_ASCII appears to be unused in MoarVM | 20:19 | |
| MVM_STRING_GRAPHEME_8 is used though | |||
| not sure why it exists | |||
| since it is never used, and its processing is the same as MVM_STRING_GRAPHEME_8 | 20:20 | ||
| and nothing ever sets strings as MVM_STRING_GRAPHEME_ASCII | |||
| MasterDuke | i think when you create small string literals from the command line they're *_ASCII | 20:25 | |
| samcv | oh i guess they are sorta different. Grapheme_8 can have synthetics down to -128 or something | ||
| nah they are MVM_STRING_GRAPHEME_8 | |||
| MVMGraphemeASCII and MVMGrapheme8 are both MVMint8, one just doesn't have negative numbers | 20:29 | ||
| or any synthetics | |||
| MasterDuke | m: use nqp; my @array; say nqp::existspos(@array, 2) | ||
| camelia | This type (Array) does not support elems in block <unit> at <tmp> line 1 |
||
| samcv | so it can still fit things into 8 bits in case there are synthetics + no characters above 127 | ||
| MasterDuke | that's a nicer error than a SEGV | ||
| samcv | ya | 20:30 | |
|
21:08
quotable6 joined,
unicodable6 joined,
evalable6 joined,
bisectable6 joined,
bloatable6 joined,
committable6 joined,
benchable6 joined,
statisfiable6 joined
|
|||
| samcv | timotimo, can you show me the workload that uses all the time in collapse_strands please | 21:20 | |
| jnthn | lizmat: If you want it for internal use, just declare it in Rakudo::Internals. :) If you want to expose it, then it'll need more discussion/thought. | 21:23 | |
| lizmat: Should be as easy as class Rakudo::Internals::YourNameHere is repr('VMHash') { } | |||
| timotimo | samcv: you have access to the whateverable server, right? | 21:24 | |
| lizmat | yeah, it's already in core atm | ||
| samcv | yeah | ||
| lizmat | jnthn: but I could still move it to Rakudo::Internals | 21:25 | |
| samcv | jnthn, how do you feel about MVM_string_graphs_nocheck because in many places we check twice | ||
| MVM_string_graphs does a check and MVM_string_check_arg does a check | |||
| timotimo | /home/bisectable/git/whateverable/irc/cache.tar.gz | ||
| grab that file and extract it | |||
| that's the 420 megs file that i'm using | |||
| samcv | ok what's the test job | 21:26 | |
| lizmat | jnthn: I called it IterationSet | ||
| samcv | i will do that in a bit | ||
| timotimo | then check out the latest JSON::Fast, and do like spurt("otherCache", from-json(to-json(slurp("cache").split("\0"))).join("\0")) | ||
| that's already run for 13 hours on my desktop | 21:27 | ||
| so you may want to cut the size of "cache" down by like 10x | |||
| or 100x | |||
| samcv | ok. it's just a long text file? or something? | ||
| are there any cp's above 127? | |||
| timotimo | it's every line of irc text from some log, but instead of newlines it has \0 | 21:28 | |
| samcv | if not, i might know what's slowing it down | ||
| timotimo | it most probably does | ||
| samcv | if there's only ascii chars in it | ||
| ok | |||
| when we colapse strands and it fits in 8 bits, after collapsing the strand, we then convert taht 32bit string into an 8bit string | |||
| Zoffix | timotimo: so JSON::Fast now uses 1/10th of the memory; did it get faster too? By how much? | ||
| timotimo | Zoffix: it only got much much faster for your problematic string | 21:29 | |
| it got slower for other strings | |||
| Zoffix | :( | ||
| lizmat | jnthn: testing whether s/IterationSet/Rakudo::Internals::IterationSet/ creates an issue | ||
| jnthn | lizmat: What are you actually using it for? But yeah, stick it in internals...I guess i's not in public API | ||
| timotimo | i don't have a good set of long json strings to test this on | ||
| jnthn | I mean, IterationBuffer plays a role in custom iterator implementations | ||
| Zoffix | Well, speed was never an issue for me with it. | 21:30 | |
| jnthn | samcv: Hm, and the C compiler ain't smart enough to avoid 'em? I guess we can | ||
| (Add one | |||
| samcv | idk if it is smart enough to avoid them. probably not because often we do other things afterward | ||
| call functions etc. | |||
| jnthn | Yeah, true | 21:31 | |
| samcv | okay cool | ||
| jnthn | Anyway, no objections if there's some kind of measurable improvement from it :) | ||
| samcv | what if there is non :P | ||
| jnthn | callgrind is probably going to be able to tell you best :) | ||
| samcv | *none | ||
| maybe it's some very small amount... | |||
| jnthn | Sure; callgrind will tell you in CPU instructions, though :-) | 21:32 | |
| lizmat | jnthn: no objections against IterationSet, or do you want it to be R:I:IterationSet ? | ||
| jnthn | lizmat: The latter, plesae | ||
| samcv | timotimo, copying that file now | ||
| lizmat | (the latter is more typing) | ||
| ok | |||
| jnthn | Yes, but exposing stuff we don't know we want to keep is less than good :) | ||
| lizmat | ok | 21:33 | |
| jnthn | Or want to call different or whatever | ||
| lizmat | fair enough, running spectest now | ||
| jnthn | I mean, I'm a bit uneasy with the name IterationSet, but if it's internal we don't need to worry ) | ||
| *:) | |||
| timotimo | samcv: looking forward to your findings :) | ||
| samcv: i didn't touch the whole topic at all after starting that | |||
| lizmat | jnthn: and it does make 5% difference on Set.new(42) | ||
| timotimo | i thought i could first figure out how long exactly it runs for | ||
| jnthn | lizmat: Nice :) | 21:34 | |
| samcv | never finishes? | ||
| how big is the file? | |||
| 400MB or whatever? that's pretty big | |||
| timotimo | anyway, yeah, 95% time spent in collapse_strands or functions called by collapse_strands | ||
| lizmat | jnthn: with one frame less and one object created less | ||
| timotimo | 420 blaze it megabytes | ||
| samcv | k i will fix your problem timotimo (maybe) | ||
| lizmat | jnthn: for each Set.new | ||
| timotimo | whoa | 21:35 | |
| jnthn | Sounds good :) | 21:36 | |
| timotimo | fwiw, all i need this for is to check if json::fast roundtrips things properly | ||
| the memory usage pattern has changed, it's now peaking up to 5.5gb every few sconds, in between going down to 2.5gb | 21:39 | ||
| Zoffix | Is that still 1/10th of previous usage? | 21:48 | |
| Zoffix just told people to upgrade JSON::Fast to get awesomification | |||
| timotimo | i hope it's not buggy | 21:51 | |
| i didn't try this workload with the previous JSON::Fast | |||
| i can imagine it'd take a bit more memory | |||
| samcv | ok timotimo callgrinding | 22:05 | |
| timotimo | how small did you make the file? | 22:06 | |
| samcv | probably not small enough idk | 22:07 | |
| timotimo | i really ought to have a corpus of json files that i can benchmark against all the revisions i've made | 22:10 | |
| samcv | gonna run it with shorter file | ||
| how long for from-json(to-json(slurp("scache").split("\0"))).join("\0")) btw | 22:11 | ||
| how long will i have to be waiting | |||
| for 120MB file | |||
| timotimo | i don't know if the time usage will decrease linearly, quadratically, or exponentially | 22:13 | |
| if it's linear, you'll have to wait a minimum of 3 hours | |||
| samcv | that is too much | 22:14 | |
| timotimo | well, yeah | 22:16 | |
| that's why i suggested going down to 1% rather than 10% | |||
| samcv | but why are we splitting and joining by "\0" | ||
| just for fun? | |||
| or is that what we're testing not the json stuff | |||
| timotimo | see if it roundtrips perfectly | 22:17 | |
| basically, someone suggested i use this file as the ultimate test for my escaping and parsing of strings | 22:21 | ||
| so i said "yeah why not" | |||
| and it turned out that it takes rather a long time | |||
| samcv | ok i made it 1/100 of the number of characters | 22:22 | |
| *graphemes that is | |||
| uhm and it's 2.5MB | 22:23 | ||
| so hopefully it doesn't take like hours | |||
| timotimo | right | 22:25 | |
| let's hope so | |||
| MasterDuke | 2mb took 32s, 186728maxresident | 22:28 | |
| samcv | not in cachegrind :S | 22:30 | |
| timotimo | oh, yeah | ||
| that makes another 2x to 10x difference? | |||
| samcv | sometimes feels like it's 1000x XD | ||
| timotimo | only when you're watching it :D | 22:31 | |
| MasterDuke | with just one data point for each way, making that change is posited in collapse_strands took 0.1s longer | 22:35 | |
| samcv | gotta be slower than 10x. it's still going after 15 mins | 22:37 | |
| so at least 32x | |||
| what good are these 16 threads now T_T | 22:41 | ||
| err 8 threads | 22:42 | ||
| timotimo | i'm bout to reboot my 'puter, so the time measurement is going to be capped here | 22:44 | |
| at 14h10m | |||
| samcv | i feel like a nap | 22:46 | |
| and when i get back it will be done :P | |||
| timotimo | i hope it will :) | 22:47 | |
| samcv | oh. no. i accidently didn't use the module | 22:59 | |
| oh well. runnig again. good thing i haven't started my nap | |||
|
23:31
agentzh joined
|
|||