Perl 6 language and compiler development | Logs at colabti.org/irclogger/irclogger_log/perl6-dev | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
Set by Zoffix on 27 July 2018.
01:28 dct_ left 01:41 softmoth joined
MasterDuke timotimo: around? 01:50
01:54 softmoth left, softmoth_ joined 01:56 softmoth_ left
timotimo for a little bit, aye 01:57
01:57 softmoth_ joined
timotimo MasterDuke: *noise* 01:58
MasterDuke re parsing, do you think there is anything not terribly complicated i could look into/
?
i've heard mention of "passing fates down" or something like that, but that seems a bit more than i want to tackle 02:00
timotimo aye 02:01
i'm actually not quite sure what exactly makes our parsing slow, or what parts contribute how much 02:02
and i'm not entirely sure how to measure it 02:03
MasterDuke a --profile-compile wouldn't help? 02:04
timotimo i'd be interested in something more granular than that
like, we already emit mvm bytecode from the regexes, but there's of course many calls in there, too
are we perhaps doing a bad job of inlining? could that have something to do with anything? 02:06
MasterDuke hm, not sure how easily i can help figure those things out 02:07
timotimo nulled.red/@flussence/101508436302890919 02:09
MasterDuke well, a perf record with a jit map shows the top function is github.com/perl6/nqp/blob/master/s...#L768-L962 02:10
which is where i did some micro optimizations, but nothing jumped out at me as a next step 02:11
well, that's good news!
timotimo ah, of course, the NFA optimizer 02:12
but that's to do with creating regexes or rather grammars
i thought we were talking about running regexen against text/data :)
MasterDuke well, i was recording `multi sub infix:<[a]>(Complex $a, Complex $b) { "$a [] $b" }; multi sub infix:<[b]>(Num $a, Num $b) { "$a [] $b" }; multi sub infix:<[c]>(Int $a, Int $b) { "$a [] $b" }; multi sub infix:<[d]>(Str $a, Str $b) { "$a [] $b" }; my $a; $a = $_ [c] $_ for ^10; say $a` 02:13
so yeah, not exactly the same thing
but also annoyingly slow
timotimo right, defining custom operators causes the grammar to be mixed into, which means NFAs have to be updated, and they also get optimized after every step 02:14
how long did that whole thing take? 02:15
i assume it was 95% compile time or something :) 02:16
MasterDuke 3s, all stage parse 02:17
timotimo holy... that's brutal
MasterDuke a perf record with a jit map of juerd's mqtt regex.t doesn't show anything terribly useful 02:18
timotimo hm, do we generate optimal code in nqp for postcircumfix:<[ ]>? i.e. for assignment and extraction, do we use the right _i, _n, _s if possible? 02:22
i'd imagine we'd always do a boxing there 02:23
so maybe that's an attack point
if $edges[0] == $EDGE_EPSILON {
though of course it could be that $edges is actually an object array
MasterDuke i think it is 02:24
timotimo oh, we could measure which of the three passes is the slowest
can you log how often the EPSILON and the FATE branch are taken, and how often no branch is taken? if the "no branch" case is the most common one, moving the "my int $to" into the two branches could save us a little bit 02:26
MasterDuke building rakudo now... 02:31
64308 EDGE_FATE 02:35
162834 NEITHER
237186 EDGE_EPSILON
timotimo wow that's a lot of epsilon edges
m: say 162864 / (64308 + 162834 + 237186) 02:36
camelia 0.350752
timotimo it could still be worth moving either the declaration or at least the/an assignment into the two branches for $to 02:37
MasterDuke oh! that may have taken stage parse from 3s to 2.3s 02:43
02:43 softmoth_ left
timotimo whoa, for real? 02:43
no way
can you try what happens if you comment out all the note("blah") if $nfadeb ? 02:44
MasterDuke not certain, on laptop and that 3s may have been when i was doing all that debug printing
timotimo oh, yeah, debug prints will slow things down a whole lot for sure
MasterDuke i think i tried that already (commenting out if $nfadeb) and it didn't make a noticeable difference 02:45
timotimo OK
if +$!states[$s] { 02:46
^- i wonder, isn't this equivalent to just if $!states[$s] ?
MasterDuke ugh, lost the original timing out of my scrollback 02:47
gotta rebuild everything...
timotimo d'oh 02:48
i wonder if your "int by default" branch does anything, too
MasterDuke i've been testing on my laptop, which isn't on that branch
could try it on my desktop which is on (and would also be more stable timings) 02:49
timotimo ah, i see 02:51
well, i'll go to bed real soon now
so no hurry from my side, though i'd love to read your results when i get back :)
MasterDuke cool, i'll mention then here 02:52
timotimo TYVM
i hope the improvement actually does exist, because that would be absolutely amazing
MasterDuke 2.2s on my branch, otherwise unmodified
timotimo oh
so perhaps the improvement was more like 0.0s
MasterDuke different system, my desktop is usually slightly faster 02:53
timotimo oh, of course
MasterDuke hm, now i'm not really seeing a change with moving the declaration inside the branches 02:56
2.2s on my branch, moved the declaration inside the branches 02:57
timotimo i hope your desktop builds quickly enough so you can give comparing the times of the three passes a shot 03:02
o/ 03:03
MasterDuke later... 03:04
aren't there 4 passes?
Hash %h = {:fifth(0.19517993927001953e0), :first(0.14771556854248047e0), :fourth(0.051312923431396484e0), :second(0.13711905479431152e0), :third(0.11382579803466797e0)} 03:21
sum of the different passes' times 03:22
03:57 leont left 07:21 ufobat__ joined 07:24 ufobat_ left 07:36 dct_ joined 09:13 lizmat joined 09:19 patrickb joined 09:24 lizmat left 09:47 patrickb left 10:13 dct_ left
[Tux] Rakudo version 2018.12-279-g92ebc333a - MoarVM version 2018.12-45-ga9d02578a
csv-ip5xs0.762 - 0.766
csv-ip5xs-206.684 - 6.753
csv-parser23.225 - 25.843
csv-test-xs-200.430 - 0.452
test7.328 - 8.985
test-t1.928 - 2.012
test-t --race0.884 - 0.886
test-t-2032.518 - 33.178
test-t-20 --race10.360 - 10.708
 10:30
10:33 lizmat joined 10:57 lizmat left 11:08 dct_ joined 11:28 gugod joined 11:40 dct_ left 11:41 lucasb joined 11:43 dct_ joined 12:04 dct_ left
gfldex I'm also getting „realloc(): invalid next size“ in IO::Handle.read. :-/ 12:06
this is not well tested
MasterDuke huh, i thought that stuff went through a round of improvement a while ago 12:25
gfldex: could you create issues?
gfldex Will do. May take some time tho.
12:28 dct_ joined
timotimo MasterDuke: aaw, the slowest one doesn't stand out as strongly 12:30
can you do a tiny bit of statistic analysis on these numbers? like, how many runs are there and how strongly do they diverge, if at all? 12:31
12:32 leont joined
timotimo it'd be cool if we could be smart about skipping some stuff in the optimizer if we had an optimized NFA already before we added some stuff 12:35
MasterDuke timotimo: Hash %h = {:fifth(235), :first(236), :fourth(236), :second(236), :third(236)} 12:57
timotimo that's how often each phase runs? 12:58
MasterDuke yeah 13:09
timotimo what's the distribution of nodes, i wonder. would probably also be interesting to count edges before optimization and after 13:11
MasterDuke just printing nqp::elems($edges) at start and end of the function? 13:12
or $!states? 13:14
timotimo let's see 13:16
$send doesn't change its value during optimization, so it's a good choice for "number of states" 13:17
the number of edges would have to be summed up from the individual state's lists
MasterDuke timotimo: sum of all the outputs? 13:37
timotimo nah 13:40
do you know what the NFA looks like as a datastructure?
MasterDuke nope
fyi, largest number of edges was 112242, occurred four times 13:42
timotimo the simplest explanation is a graph
MasterDuke yeah, makes sense 13:45
timotimo i would normally say "just graph it with graphviz", but ... 13:47
112k nodes
not so fun to look at
oh, edges?
MasterDuke yeah. added this at the very end of optimize 13:49
gist.github.com/MasterDuke17/dfedd...7d791611fa
timotimo that looks correct 13:52
what send / edges value combinations do we get?
MasterDuke 109 distinct 13:53
want to see them all? 13:54
timotimo sure
what'd also be interesting to see, but more work to measure, is how many distinct edges there are actually 13:55
i.e. how many distinct (source-state, target-state) edges there are, maybe also how often each of those occurs
maybe it is possible to graph that stuff actually
if multiple edges between the same states get collapsed into one 13:56
MasterDuke gist updated 13:58
is that what the dup edge remover does?
timotimo not quite
the kinds of edges we have are things like "character class" or "one specific character" 14:00
so perhaps there's an edge for "any digit" and also an edge for "☺", those can't be deduped sensibly
MasterDuke oh, are you talking about removing multiple edges to the same state just for visualizing? 14:03
timotimo yeah 14:04
it's not a viable optimization i don't think
MasterDuke ah
AlexDaniel timotimo: just 112k nodes, still “just graph it with graphviz” :)
timotimo though perhaps we could more closely examine what kinds of same-edge-different-kind we encounter
AlexDaniel not that it will produce a readable graph, but you can still graph it for sure :P 14:05
MasterDuke would it make sense to also put that code at the beginning to see how many edges are actually removed in total?
timotimo and see if there is some optimization where we can collapse some kinds
MasterDuke: that could be interesting, though perhaps instead log how often the individual phases "do something" 14:06
AlexDaniel: i recently had some program that's made for very, very large graphs
AlexDaniel timotimo: fwiw I learned to love graphviz when I was using yosys daily. It has some extra simple functionality on top that makes it so much easier to look at the graphs 14:09
14:09 dogbert17 left
AlexDaniel timotimo: one is random coloring of edges. I wonder why graphviz can't do that natively, makes life so much easier on crowded graphs 14:10
timotimo: and another is that it has commands to select parts of netlist of your interest. So “I wanna look at X, graph things around it” and so on
timotimo ah, so it just generates new graphviz documents for those cases 14:11
AlexDaniel yeah 14:12
maybe it's a good idea for a standalone program
dot file in, select some nodes, dot file out 14:13
maybe it exists already?
timotimo i'd like more support for changing stuff in an existing graphviz-ed graph
if you re-graph after changing, chances are that nodes turn up in completely different places 14:14
AlexDaniel definitely, yes
timotimo though you can ask graphviz to output its exact layouting back at you as a dot file 14:15
MasterDuke gist updated
14:15 dct_ left
timotimo so a 50% reduction isn't uncommon 14:16
MasterDuke seems worth it 14:18
timotimo hm, do we have the "name" of the nfa in question on the stack? might be interesting to compare what kind of evolution one nfa goes through as more stuff gets added 14:20
MasterDuke # The NFA we will actually run (with NFA REPR). has $!nfa_object; 14:22
timotimo well, that's the nfa object, but i believe it gets recreated after changes 14:23
there's an nfatostatelist and an nfafromstatelist op
and the nfa object itself can't be modified
MasterDuke the REPR doesn't look like it has a name or anything like that 14:24
timotimo right, we'd have to find a name for the thing that has it 14:25
anyway. is it really important? not sure ... 14:27
14:55 dct_ joined 15:11 lucasb left
Geth rakudo/argfiles: cd3fd90d3e | (Tom Browder)++ (committed using GitHub Web editor) | src/core/Argfiles.pm6
Describe creation of a new dynamic variable @*ARGFILEPATHS
 15:18
tbrowder It’s just a comment for a reality check for a possible new dynamic variable. I would appreciate any criticism. 15:20
Geth rakudo/argfiles: 4f28185f6a | (Tom Browder)++ (committed using GitHub Web editor) | src/core/Argfiles.pm6
finish the proposed replacement code
 15:27
AlexDaniel tbrowder: I've been looking at this discussion on and off and honestly I don't understand what problem you guys are trying to solve 15:36
leont doesn't really understand it either, TBH 15:37
AlexDaniel is there a ticket somewhere?
R#2666 15:38
synopsebot R#2666 [open]: github.com/rakudo/rakudo/issues/2666 implement a new var @*ARGFILES listing paths for @*ARGS that are existing files
AlexDaniel tbrowder: so let me get that straight…
user runs the script like `foo.p6 file1 file2 file3` 15:39
one of the files doesn't exist, so it fails
for me it makes sense. Now what are we trying to solve?
tbrowder saving having to produce my own list of readable fike 15:41
*files from *args
my new code shouldn’t fail, it skips unreadable args 15:42
worst case should be ampty @*ARGFILEPATHS
leont I can't imagine wanting that behavior, TBH 15:43
AlexDaniel tbrowder: so why can't you add a CATCH block? 15:44
like
for lines() { .say }; CATCH { default { } }
evalable6 ♥🦋 ꒛㎲₊⼦🂴⧿⌟ⓜ≹℻ 😦⦀🌵 🖰㌲⎢➸ 🐍💔 🗭…
AlexDaniel, Full output: gist.github.com/84814d4870c8f8f31d...7b9f02b18a
AlexDaniel … thank you evalable6…
I see a clear problem that the exception in that case is AdHoc 15:45
tbrowder ok, can’t that be handled graceduk 15:46
fully?
got to go...back in an hour or so...bye 15:47
AlexDaniel R#2669 15:51
synopsebot R#2669 [open]: github.com/rakudo/rakudo/issues/2669 [6.e] Remove all AdHoc exceptions from core 15:52
15:53 leont left
b2gills Currently `lines()` will generate an exception, I think it should pass a Failure..3 16:14
16:16 Mediphira joined, Mediphira left
AlexDaniel b2gills: how is that supposed to work?? 16:20
honest question 16:21
so you have a loop, say `for lines() -> $line { }` 16:22
b2gills It might be as simple as adding a check to the `lines` iterator, and have it return the error through `pull-one`
AlexDaniel so you get a Str in $line a bunch of times, and then suddenly you get a Failure? 16:23
I wonder how that is going to affect lines().elems or the like
btw I tried .resume and it said “This exception is not resumable” 16:28
b2gills m: sub foo (){ Seq.new: class :: does Iterator { has $!i = 0; method pull-one() { ++$!i; if $!i ==6 { return IterationEnd } elsif $!i==3 { fail 'Some Failure' } else { return $!i }}}.new}; for foo() -> $line { $line.perl.say } 16:32
camelia 1
2
Failure.new(exception => X::AdHoc.new(payload => "Some Failure"), backtrace => Backtrace.new)
4
5
tbrowder leont: i want to do something like: “for @*ARGS ->$a { next if !$a.IO.f; say “arg $a is a readable file”. 16:33
b2gills Add the functionality to skip erroneous files to $*ARGFILES then you can do `my @FILES = $*ARGFILES.handles` 16:34
tbrowder of course it would have to be protected i guess from bad actors, but if i can do it currrently with @*ARGS, why not with an @*ARGFILES?
that would work, but still gives me an unwanted handle i would have o get a path from. 16:36
forget for now whether we need it or not, would my argfiles branch suggestion work as it is? 16:38
i make an assumption that the $argiter is an nqp string. 16:39
so that may need some tweaking...
AlexDaniel tbrowder: no, the proposed change is not going to work, but we can think of other ways we can make it possible to do what you want it to do 16:46
why – because you're trying to do some checks on the file ahead of time, even though the file will be accessed much later (maybe even hours after you checked it) 16:47
so you still have to handle exceptions/failures when reading it, meaning that the check is worthless 16:48
16:57 dct_ left 17:08 dct_ joined
tbrowder i must admit i’m not thinking about long-running prospcesse :-( 17:09
*processes 17:10
but again, i just want args that, at program init, are paths to readable files. throwing later is fine if i don’t check existence when i’m ready to use them. 17:13
i do not want the handles at init time! 17:14
i will try to get them later when i’m ready to use them. 17:15
b2gills So you are trying to get out of writing: `@*ARGS.grep(*.IO.f)` by making the already large codebase larger. 17:17
17:18 dct_ left
tbrowder i didn’t know we were trying to be another C. 17:19
p6 is already a large language, so what’s another thp
*dozen lines? 17:20
imho, i would dump $*ARGFILES for @*ARGFILEPATHS. 17:21
b2gills The only reason $*ARGFILES exists is because of the behaviour of <> in scalar context in Perl5. Otherwise it probably wouldn't have been added. 17:43
17:53 b2gills left 17:57 b2gills joined 18:40 Ven`` joined
AlexDaniel b2gills: yeah I've got the same feeling 19:25
tbrowder probably right, i rarely ever used it... 19:28
19:35 vendethiel- joined 19:37 Ven`` left 21:37 vendethiel- left 22:24 sivoais joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!