samcv hmm timotimo split on the \ for what reason 00:07
or split on the \r\n's? 00:08
timotimo json has many escape sequences
samcv yes 00:09
timotimo if i split on \ i get the pieces that will be of interest, and each string after the first will let me just inspect its first character to see what i have to join it with
samcv it could be a lot of segments
timotimo it'd be a bit awkward to do the rest of the parsing, though if i just make the $target "is rw", too, and keep a "original-pos" parameter for error reporting ...
samcv m: "\t".uniprop('NFG_QC').say 00:10
camelia N
samcv m: "a".uniprop('NFG_QC').say
camelia Y
samcv hmm
ok i might be able to improve this
in nfg_is_concat_stable
timotimo cool 00:13
but \t isn't very common in the json we have, i don't think
samcv yeah. but it could be very very common in some
timotimo of course 00:15
anything to make more concats stable (or rather make the stability easily and quickly assessible)
00:26 pyrimidine joined
MasterDuke timotimo: did you figure out if J::F is doing the "correct" number of joins? is it doing more than it needs to? 00:29
samcv i think i might have solved it. we'll see. haven't tested yet 00:31
timotimo i didn't check that 00:43
cool
i'm not sure if we have a whole bunch of concat-with-combiners tests?
samcv yeah it's undertested if at all 00:45
i mean some, but need way more since there's so many different factors…
in normalization and blah blah
timotimo mhm 00:46
MasterDuke timotimo: btw, had you seen bloaty mcbloatface before? seems like something you could make use of 00:47
timotimo i've seen it, yeah 00:53
it's mostly interesting for our unicode database
other pieces of data we have include our repr structs maybe
those are tiny, though
MasterDuke ah, cool
timotimo we can definitely run it on libmoar.so of course
samcv eek MVM_unicode_codepoint_get_property_cstr strikes again 00:55
in passes_quickcheck_and_zero_ccc. can speed that up by using int
timotimo nice 00:56
MasterDuke timotimo: re MVMNativeRefREPRData, what's involved in adding a field/flag to it? just changing the struct def in src/6model/reprs/NativeRef.h?
timotimo that's enough, yeah
samcv ok i reduced it from 3.14s to 1.89s
\o/
timotimo what benchmark is that?
samcv uh that json file
timotimo what exact part? 00:57
samcv time perl6 -e 'use JSON::Fast; my $j = "mem_leak2.json".IO.slurp; for ^1 { $ = from-json $j;} '
timotimo is mem_leak2.json smaller than the original one?
does it only have that long string in it?
samcv it's basically the same
but missing a few controls
timotimo huh, okay 00:58
samcv but MasterDuke said there was 0 difference between them
MasterDuke samcv: any change in memory used?
samcv yeah same time for the original one
haven't checked
timotimo ugh, i don't have the gist open any more
samcv but i'm sure
what's the easiest way to see max memory usage 00:59
timotimo just run it w/ time
MasterDuke /usr/bin/time
samcv how does that get me momory
MasterDuke just `time` is usually a shell builtin
samcv yeah
i don't get memory stats from time 01:00
timotimo right sorry
MasterDuke have to install time from extra
samcv not on bash either
cool
timotimo fish just gives you the right kind of time stuff by default
samcv 1.94user 0.52system 0:02.47elapsed 99%CPU (0avgtext+0avgdata 2183472maxresident)k 01:01
0inputs+0outputs (0major+540131minor)pagefaults 0swaps
what am i looking at. maxresident?
MasterDuke yeah, i was getting 2180912maxresident before 01:02
samcv ok from 2621932K to 2183472k
and even bigger time difference
01:10 pyrimidine joined
samcv i'm putting all my notes for unicode here github.com/MoarVM/MoarVM/projects/1 01:16
so i can make sure i have it all down in one place that other people can see as well.
timotimo i saw 01:19
maybe i want to build such a project thing, too 01:20
samcv :)
timotimo for example for the line-based profiler
samcv sounds like a good plan
looks like you can even convert them to issues as well 01:21
01:23 ggoebel joined
timotimo i noticed 01:23
samcv i just discovered that now
timotimo i'm surprised trello isn't mad at them for stealign their thing
samcv just gotta spectest the concat optimizations 01:29
01:37 pyrimidine joined
samcv ok well spectest is pass timotimo 01:46
json_fast fails test tho
nvm it fails them on master too 01:48
wtf 01:49
were they already failing though?
both fail same number of tests. timotimo 01:51
i mean with and without change. and even reverting before MasterDuke's changes
timotimo what test is failing?
samcv t/05-unreasonable-requirements.t and 06-control 01:52
only those
timotimo oh? 01:53
samcv yep
timotimo let me check
o 01:54
are you making sure to -Ilib?
both pass for me 01:55
samcv oh. yeah that prolly 02:00
how do i ilib timotimo 02:02
MasterDuke `perl6 -I lib t/05-unreasonable-requirements.t` 02:06
samcv k 02:08
nope still failing
i did prove -e perl6 -I lib
ok running it manually is fine tho 02:09
timotimo weird. 02:17
i don't think that's going to work what you did there
it'll be parsed as -e perl6 -I lib
rather than -e "perl6 -I lib"
-l, --lib Add 'lib' to the path for your tests (-Ilib). 02:18
though that would have worked
huh, prove can also handle -I apparently
MasterDuke `prove -e perl6 -I lib` fails 05 and 06 for me, but `prove -e "perl6 -I lib"` passes them all 02:21
timotimo well, prove doesn't give it a -I 02:22
it also doesn't put anything in the environment 02:23
so ... ?!?!
prove introduces a separation between "options passed to the perl which runs prove" and "options passed to the perl which runs tests"; this distinction is by design. Thus the perl which is running a test starts with the default @INC. Additional library directories can be added via the PERL5LIB environment variable, via -Ifoo in PERL5OPT or via the -Ilib option to prove. 02:24
^- this doesn't tell me whether -Ifoo will change what the prove process is using or what the called process is running 02:25
don't want to go on a deep dive through the guts of prove to find out if -l or -Ilib should have worked 02:28
02:48 ilbot3 joined
samcv timotimo, ok so my changes appear to work fine for at least simple examples and spectest pass 04:13
like concatting things that change under NFC for example like letter + diacritic. and \r\n is fine
so i think it's good
Geth MoarVM: samcv++ created pull request #558:
Optimize MVM_nfg_is_concat_stable to speed up concatenation of strings		 04:52
samcv should be ready for merge. was thinking about being more agressive with the optimization for \r and \n and not forcing normalization if the other character is 'unspecial'. but for now that is good 04:54
06:32 brrt joined 07:06 domidumont joined 07:12 domidumont joined
Geth MoarVM: 477e023bd8 | (Samantha McVey)++ | src/strings/nfg.c
Optimize MVM_nfg_is_concat_stable to speed up concatenation of strings

Here we special case '\r' ~ '\n' to ensure it will be normalized, since it is a common possible case.
All other characters that have Grapheme_Cluster_Break=Control will always break and do not change under normalization. ... (6 more lines)		 07:20
MoarVM: 029d121826 | (Samantha McVey)++ | src/strings/nfg.c
Merge pull request #558 from samcv/nfg

Optimize MVM_nfg_is_concat_stable to speed up concatenation of strings
07:37 travis-ci joined
travis-ci MoarVM build errored. Samantha McVey 'Merge pull request #558 from samcv/nfg 07:37
travis-ci.org/MoarVM/MoarVM/builds/213726983 github.com/MoarVM/MoarVM/compare/c...9d12182633
07:37 travis-ci left
samcv how 07:37
it timed out. restarted the build 07:38
07:42 brrt joined 07:43 domidumont joined
Geth MoarVM/even-moar-jit: 4f4c5f2768 | (Bart Wiegmans)++ | 3 files
Use macro's to extract tile inserters

It's no good to pass half the state of your function to a callee if you can do the same within your own scope (and shorter, too).		 08:29
MoarVM/even-moar-jit: dafb674f7b | (Bart Wiegmans)++ | 4 files
Improve prepare_arglist_and_call toposort

Adding and removing edges is now much more explicit (spills and arglist/call conflicts are now counted as outbound edges); renamed the variables to make the underlying concept more clear.
brrt after i've split ARGLIST and CALL tiles again (which is more or less a simple translation), this should hopefully be correctish 08:50
samcv o/ brrt 08:52
brrt \o samcv 08:53
09:07 zakharyas joined
lizmat .tell jnthn I left some comments with gist.github.com/lizmat/b76e88519b7...0941bb6e6a 09:36
yoleaux2 lizmat: I'll pass your message to jnthn.
brrt \o lizmat 09:51
lizmat brrt /o 09:52
samcv i think i'm going to bed. night all :) 09:53
brrt sleep well
lizmat nighty night, samcv!
good work! 09:54
brrt lizmat: it almost looks like you're writing in lisp there 09:58
lizmat ah?
brrt nqp::while(nqp::islt_i(), nqp::if(…) etc 10:01
that'd be spelled (while (is_lt ..) )
in lisp :-)
lizmat ah, in that sense :-)
yes, there are some visual similarities :-) 10:02
brrt it is horribly complex to read, though..... 10:06
10:46 Geth joined, yoleaux2 joined
lizmat .tell jnthn a new gist: gist.github.com/lizmat/1c80d2210ae...18fca7e575 11:18
yoleaux2 lizmat: I'll pass your message to jnthn.
lizmat .tell jnthn this version created R:It.(Hyper|Race)Actions iterators, that take a list of method => capture pairs to execute: so no special casing needed anymore 11:19
yoleaux2 lizmat: I'll pass your message to jnthn.
lizmat .tell jnthn from where I sit, we only need to fix the syntactic sugar for transforming .hyper.map(foo).grep(bar) to R:It.HyperActions( , (map => \(foo), grep => \(bar) ) 11:21
yoleaux2 lizmat: I'll pass your message to jnthn.
lizmat moved/adapted comments to new gist and removed the old gist 11:39
12:37 domidumont joined 12:46 domidumont1 joined 14:47 domidumont joined
brrt lizmat: re your comments below, the big thing with concurrent software is always, what happens when a process dies 15:27
lizmat you mean when a worker has an exception ? 15:28
brrt uhuh
or waits forever
lizmat then it will wait forever 15:29
unless you have a promise waiting somewhere that will provide relief 15:30
brrt true….
my head just says 'condition variable' when you write 'count how many IterationEnd we should get'… but I guess if you have monitor semantics anyway, it's not going to be a problem 15:31
lizmat I'm trying to avoid using explicit condition variables 15:33
using the ConcBlockingQueue repr for that, basically 15:34
brrt indeed 15:36
so probably… if any one of the proceses dies, the whole thing should be killed? 15:37
timotimo how about letting the developer catch that case?
lizmat if your .map has a CATCH, you could :-) 15:38
brb
brrt i'm smelling danger
(i'm heading to $commute)
timotimo doesn't it have to be outside the map? 15:39
16:04 brrt joined 16:14 ZofBot joined 16:18 SourceBaby joined
brrt omg omg omg 16:39
it works :-o
jnthn The JIT? :) 16:40
Calls?
:)
16:41 ZofBot joined
brrt yes 16:43
c function calls 16:44
can't say i've properly tested every bit of it
because i haven't
but
call ordering and spilling and restoring works
jnthn :-)
That's a rather important piece :) 16:45
16:47 Zoffix_ joined
brrt yeah 16:47
Geth MoarVM/even-moar-jit: ef816962b9 | (Bart Wiegmans)++ | 3 files
Split ARGLIST and CALL tiles

This makes the very simple 'say' function call work as expected. Note that not all functions of the prepare_arglist_and_call are completely tested as of yet (that's only possible with more complex calls). Basic parameter ordering and spilling/restoring works.		 16:50
brrt i think… with that done, it's time for me to start working on a blog post again
jnthn :) 16:51
brrt++
brrt oh and there's another thing i need to resolve, which is generalised two-arg resolution 16:52
16:59 domidumont joined 17:41 domidumont joined 17:53 synopsebot6 joined 17:55 lizmat joined
timotimo yaaaay 17:56
Geth MoarVM: dc974a7353 | (Samantha McVey)++ | .travis.yml
[Travis CI] Add travis_retry to apt-get lines to retry package dl probs.
 20:41
MoarVM: samcv++ created pull request #559:
Optimize passes_quickcheck_and_zero_ccc to be much faster
samcv mmm tea 20:43
20:45 spebern joined 20:46 travis-ci joined
travis-ci MoarVM build errored. Samantha McVey '[Travis CI] Add travis_retry to apt-get lines to retry package dl probs.' 20:46
travis-ci.org/MoarVM/MoarVM/builds/213998643 github.com/MoarVM/MoarVM/compare/0...974a735349
20:46 travis-ci left
Geth MoarVM: a2367a2097 | (Samantha McVey)++ | .travis.yml
[Travis CI] Don't use travis_retry and instead just use ||
 20:52
samcv weird got `/usr/bin/ld: src/core/coerce.o: relocation R_X86_64_PC32 against undefined symbol `fast_atoi' can not be used when making a shared object; recompile with -fPIC` 21:02
gonna try make clean
timotimo hm, we don't -fPIC by default? 21:03
it's definitely in my makefile 21:04
samcv we do.
timotimo builds bloaty 21:05
um, apparently i already have bloaty 21:06
somewhere on my machine
found it 21:08
when broken down by compileunit, the vast majority of size is in "[None]" 21:11
samcv ok i think i fixed it 21:13
Geth MoarVM: 907cbc7722 | (Timo Paulssen)++ | 44 files
give this_repr consts more meaningful names

they show up when you print an object in gdb, for example		 21:28
MoarVM: f12a2b5351 | (Daniel Green)++ | src/strings/ops.c
Use correct format for repeat/concat errors
 21:32
MoarVM: 61f02f81c7 | (Jonathan Worthington)++ | src/strings/ops.c
Merge pull request #555 from MasterDuke17/master

Use correct format for repeat/concat errors
MoarVM: ddc97f1075 | (Samantha McVey)++ | src/strings/nfg.c
Optimize passes_quickcheck_and_zero_ccc to be much faster

Use MVM_unicode_codepoint_get_property_int instead of MVM_unicode_codepoint_get_property_cstr and simplify it as well.		 21:53
MoarVM: d866829cac | timo++ | src/strings/nfg.c
Merge pull request #559 from samcv/nfg_2

Optimize passes_quickcheck_and_zero_ccc to be much faster
timotimo i took the liberty
samcv thx timotimo 21:58
Geth MoarVM: 1e2a7b90cd | (Samantha McVey)++ | 4 files
Streamline MVM_bigint_radix and MVM_radix for Nd Unicode #'s

We can use the fast_atoi function in addition by switching to using Numeric_Value_Numerator instead of Numeric_Value (which may have decimal places). This is because all Nd numbers are 0-9 and thus all have a denominator of 1.
timotimo oh, neat 21:59
samcv no speed difference. but change i'd thought about making before
timotimo OK
samcv no measurable one at least
timotimo that's also only for "fancy" digits, right?
samcv yeah 22:00
also when i redo the UCD, i'm going to use the Numerator property instead of the Numeric_Value property as well, so thought i might as well change it even if no speed change
will probably get some speedup with the remake becuase it'll be an int proprety and won't have to atoi it. 22:01
and Numeric_Value property is probably going bye bye
at least if it doesn't hurt anything 22:02
since it's just a string that holds a floating point number
timotimo ugh :)
samcv heh
timotimo comp_l_* are for unicode comparison stuff?
samcv what's comp_l_*? 22:03
timotimo a range of symbols i found with bloaty
samcv let me see what it does :P
timotimo oh, they only go from 0 to 21
maybe more interesting is comp_entry_0 through 375 22:04
samcv oh canonical composition tables
timotimo ok, they are all rather small, so they are probably packed tightly 22:05
12 bytes for every comp_entry_*
and 12 for every uni_seq_*
samcv yeah
ok i added uni_seq at least 22:06
timotimo except a few that are 16 bytes
oh, and a bunch that are 20 bytes
(and so on)
also 20-byte comp_entry_*
and 24 byte uni_seq_*
*shrug*
samcv argh and the comp_entry lines reorder on rebuilds
so i can't git log -L very well
timotimo biggest two comp_entrys are 156 bytes 22:07
ugh, can we somehow sort 'em?
samcv by size?
or just to make rebuilds not change order 22:08
and make me go insane
timotimo not change order
samcv ok just make it reproducaable?. also curious if ordering could affect packing or if the compilier does that automatically 22:09
at least for seperate arrays
timotimo absolutely no clue 22:10
if nm is right, the ordering is lexicographic 22:11
so 0, then 1, then 10, then 100, then 101, 102, ..., 11, 110, 111, 112, ..., 12, 120, ...
no, hold on
the addresses to the left of that are kin of scrambled 22:12
they are actually put in there in reverse, but in proper order
i.e. 00000000002ea150 r comp_entry_101
00000000002ea160 r comp_entry_100
00000000002ea170 r comp_entry_99
00000000002ea180 r comp_entry_98
00000000002ea190 r comp_entry_97
samcv k 22:13
installing bloaty myself 22:18
22:23 Zoffix_ joined, hoelzro joined, leego joined
samcv timotimo, what command to get all this info you're. sharing with me 22:27
timotimo one was bloaty -d symbols -n 0, the other was nm | sort 22:28
it's important to -n 0 or else you won't get the relatively tiny symbols
samcv 7.7% 407Ki codepoint_names 22:31
not sure how it stores in 407Ki?
based on my estimate we'd have used a lot more. but maybe it does some packing when it does it in a .so file
so that could be
timotimo is codepoint_names still a list of strings? 22:32
in that case it'd have one pointer per entry
samcv yeah i guses it doesn't count that? 22:33
it only counts data and not pointers?
timotimo well, the symbol codepoint_names only has the pointers 22:34
the individual strings must show up somewhere else
samcv ah ok. that makes sense 22:35
timotimo perhaps with an entry_* name or something? or are they truly anonymous? 22:37
well, i don't see 'em anywhere
i suppose because nm only shows things that have names
and those strings don't have symbol names to refer to them by 22:38
samcv yeah they don't
they're just in a big array
timotimo that explains it, then 22:39
22:39 leedo joined
samcv still unhelpful if it doesn't show what actually uses space 22:39
not just the uh. things that use space but the uh
pointers and stuff
timotimo hm, shouldn't it show up as a lot of usage for compileunit unicode.c? 22:43
samcv dunno 22:45
timotimo gonna go sleep 22:56
have a good one!
oh 22:58
samcv night
timotimo gist.github.com/cygx/9c94eefdf6300...e-graph-md - i wonder if we can figure this out together some time
maybe even find the bottleneck to one of these benchmarks and troll this graph by making it flatline :P 22:59
o/
22:59 Geth joined 23:04 ZofBot joined, yoleaux2 joined 23:09 Zoffix joined
samcv that benchmark is not fair 23:22
%words = ($input ~~ m:g/(<-[\n\t]>+)\t(\N+)/).map(*>>.Str.Slip); when perl 5 version is much simpler
perl 5 is %words = ($input =~ /([^\n\t]+)\t(\N+)/g);
goes to 7.57s from 13.27s when i remove all that extra stuff. but i guess they're wanting the values to be strings or whatever 23:32
i think it's a bad benchmark 23:33

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!