🦋 Welcome to Raku! raku.org/ | evalbot usage: 'p6: say 3;' or /msg camelia p6: ... | irclog: colabti.org/irclogger/irclogger_log/raku
Set by ChanServ on 14 October 2019.
kleb hi everybody! 06:32
i am having some troubles with the performance of a raku grammar, and was wondering if anybody here might know of any tricks or techniques i can use to figure out what is going wrong 06:34
i am basically porting on of the antlr grammers i found on github to raku
for c++
kleb if i can get this working, hopefully i will have some new tools i can use to aid in porting pieces of the c++ program i have to rust 06:35
what i want to do is parse snippets of c++ with a raku grammar
c++ has a rather large grammar
kleb i think i ported the c++ grammar properly , but now the performance of my program is prohibitive 06:37
*when i say i ported the rules properly, what i mean is that i think they are *logically* correct (or correct enough)
but now raku is slow and i am not sure how to figure out why 06:38
it is slow to compile the grammar on the first run, but even once compiled, it still takes almost a second to load the grammar at runtime 06:39
(the first run compilation takes nearly 12 seconds) 06:40
but now it has been parsing a simple c++ function declaration for 20 minutes
jmerelo Wow
kleb yea i know haha 06:41
jmerelo kleb: these things can be slow, but also make sure you're using ratcheting correctly.
kleb good point -- right now nearly everything is a 'rule' 06:42
moon-child kleb: if it took that long to compile it might just be slow, but I would try using the grammar tracer to see if it's gotten stuck in an infinite loop 06:43
you have to watch out for left-recursive rules in raku grammars that other parser generators can sometimes deal more gracefully with
jmerelo moon-child: that was my next piece of advice, yes.
moon-child :)
kleb thank you both! 06:44
"watch out for left-recursive rules" -- great point, i indeed realized this earlier today. i think i have mostly mitigated the problem, but there were a few i couldnt quite figure out 06:45
i am not sure if these are the problem rules
i didnt know about Grammar::Tracer -- that will be my next stop
jmerelo kleb: you might want to check out moritz's book on Grammars... and his articles too. It might be available in your nearest university library, or you can order it. 06:47
kleb "Raku Fundamentals"? 06:48
jmerelo kleb: www.apress.com/gp/book/9781484232279, this one is specifically about regexen and grammars 06:50
kleb ah, apress says ' Page not foundSorry, the page you requested is unavailable. The link you requested might be broken, or no longer exists.Why not start at our homepage?" 06:51
this would indeed be useful to me
jmerelo kleb: maybe this one? www.springer.com/de/book/9781484232279 06:52
kleb i could use an in depth treatment of this subject -- i know a lot and have read the pods, but i am looking to ramp up even more if possible
bingo
thank you for your help
jmerelo kleb: my pleasure
kleb i think it is almost certainly that there is a simple bug in my grammar somewhere because of a left recursion (or something similar) 06:53
i may just have to binary search comment uncomment until i can hone in on it 06:54
it is a bit tricky though because there are zillions of grammar rules for c++
i will read this book and then see if i can figure it out 06:55
moon-child you might have a better time building up the grammar iteratively, testing it at each step
jmerelo kleb: that's going to be slow no matter what... But slow and doable.
kleb yeah.. am hoping to not need to spend days writing a c++ parser -- it looked like i might be able to drag and drop the antlr right into raku and be more or less in the endzone 06:57
kleb basically i want something that i can just call right from vim on a C++ function that parses the function, translates the function declaration into rust and block comments the whole body 06:58
jmerelo kleb: that would be cool :-)
kleb yeah i know right! haha it would be sweet 06:58
kleb i dont think at this point i want to spend the time to translate the whole function body, but i think it should be do-able to just make it translate the function definitions automatically 06:59
i have been continually geeking at how well raku and rust interop btw 07:00
another idea i have been working on is how to get a rust macro to invoke a raku grammar transformation automatically
i have a basic skeleton of that working 07:01
its freakin sweet
it is basically like having a rust macro, except instead of calling the rust pattern matching code, it calls raku as a parser and code generator seamlessly 07:02
i hope one day soon i can get it so that the macro automatically has the exact name of one of the raku files in a specific directory of my project 07:03
kleb that way, adding more of those files equates to adding more rust macros that are supercharged with raku code-writers 07:03
the projects eventually just write themselves 07:05
literally
the idea is that you require an input pattern which follows a certain syntax and the name of a raku file (which is seen from rust as a macro invocation), then during rust compilation the raku grammar will parse the input pattern and write the rust code 07:08
as a step *before* the rust compiler checks the code for all the usual stuff 07:09
aite g2g thanks for the help everybody :)
kleb hi everybody! does anybody know if it is possible to attach a variable block to a method programmatically during the class definition? 08:20
moritz kleb: what's a "variable block"? 08:23
kleb looking to do something like *this* pastebin.com/raw/B8ZZXfp6 08:24
<Digitsequence><Floatingsuffix> are defined in my CppLexerGrammar class 08:25
i basically want to write an array @tests with a bunch of test cases mapping a string to the body of a "TOP()" grammar rule which should be able to parse it 08:26
kleb ordinarily, <Digitsequence><Floatingsuffix> would be verbatim the body of the TOP() rule within the grammar 08:26
im looking for a clean way to programmatically insert the body of a function into a grammar definition defined in a loop 08:27
is this possible?
moritz functions don't make much sense, grammar rules/tokens/regexes are methods 08:28
you can add methods to an existing grammar, either through the MOP or by mixing in a role (though I don't think you can use a runtime-supplied method in a role) 08:29
but somehow I don't think that's the best way to resolve your testing desires
kleb which MOP hook can perform this? I dont mind if the loop where I hook the TOP rule into the grammar is relatively ugly provided @tests is as clean as possible -- i anticipate this array having many entries in the near future 08:31
moritz .^add_method (and in the end you need to call .^compose)
kleb brilliant
thank you!
moritz but if you want to test individual grammar rules, there's a much simpler way 08:32
YouGrammar.parse($input, :rule($name))
kleb ahh, wonderful! One thing though is that the blocks dont necessarily correspond to individual grammar rules 08:33
moritz why not?
*another* option: you could formulate those blocks as rx{ ^ <CppLexerGrammar::DigitSequence> <CppLexerGrammar::Floatingsuffix> $ } 08:35
then you don't need to modify the thing you are testing
kleb I am trying to figure out how to phrase the answer to your question 08:36
<DigitSequence><Floatingsuffix> doesnt necessarily show up together in the grammar as an individual rule
moritz then why test them together? 08:37
kleb but I might want to use the grammar as a whole to match this pattern <DigitSequence><Floatingsuffix> on some text
i am writing the grammar so i have a toolkit i can apply to new and unknown future c++ parsing problems i might have
moritz if you use the grammar as a whole, it shows up in TOP, no?
maybe your testing approach could instead be: I want my grammar to match my test strings, and I also want these two rule names to appear in the match tree 08:38
kleb right now I have a role CppLexerGrammar (which is actually a role and not a grammar) -- when I want to parse I create a grammar which does CppLexerGrammar, and I write a TOP(){ <my_desired><combination><of_patterns><from_the_role> } 08:39
moritz or even: exactly these two rule names appear in this order as top-level matches
kleb I am basically looking for a solution which lets me compose just the roles I need and match arbitrary combinations of rules
moritz ah, it's not really a grammar, just a collection of related rules 08:40
kleb without having to know which combinations of rules I need at the time the grammar is written
well, the toplevel G is a grammar
and I might ultimately have a toplevel grammar which is more steady
but yes, I am collecting related rules
there is one for the cpp lexer, the cpp operators, the cpp keywords, the cpp template system, cpp classes, etc etc 08:41
probably 7 or 8 related sets of rules that overall create the requisite set of rules for a full c++ grammar
i am decomposing the problem and testing each role one by one 08:42
but i am going to need a test harness to do this properly :)
but then, later on, once this is all done, I don't exactly know what i am going to want to parse with it 08:43
for example ,I might want to parse several functions and a c++ template class
but a rule for this is naturally not found in the overall grammar ,because the grammar doesn't pin it down that specifically
but the grammar rules dont need to change
and i dont want to add more grammar rules 08:44
once i know it properly parses all known c++ translation units
that should be the point where i stop writing rules
but then i might want to compose those rules into some sort of new arrangement for an AST building task in the future
kleb if i have some initial idea about the structure of my input data, i would like to be able to draw from this fully funcitonal C++ grammar library to do my task 08:45
thank you so much for you help so far BTW -- this has been incredibly helpful for me
moritz you're welcome
kleb you have given me much to think about 08:46
moritz and I seee jjmerelo already pointed you to my book, so no need to repeat that :D
kleb yes, thank you! 08:48
i will buy on amazon!
happy to support you
am thankful for everyone who contributes to raku-land 08:49
kleb order placed! thank you! 08:50
moritz once you've read it, a short review on amazon would really help 08:52
kleb sounds good, I will! 08:53
I am grinning ear to ear because I am about to own a programming book from someone who just helped me directly! sounds simple but how cool is that!
moritz enjoy :-) 08:57
I felt the same when Larry Wall first answered me a design question on IRC
tadzik :) 08:58
kleb :) man, its times like these that make remember how much i love programming 09:00
kleb out of curiosity, do you remember what the design question was? what did he chime in about? 09:02
i think it is incredible how somebody can design such a think and then have it evolve into this wizard tool which has powered generations of programmers pushing all frontiers of human capacity 09:05
*such a thing
learning these systems has evolved my whole linguistic capacity in ways i could have never before predicted 09:06
kleb BTW: this solved my earlier issue: pastebin.com/raw/QyJv2GfU 09:14
I used your tip about rx{ }, which I use instead of the Block I had before 09:15
thanks!
Geth advent: eed1cedab4 | Altai-man++ (committed using GitHub Web editor) | raku-advent-2020/authors.md
Take back a promise for two articles

Resources are too scarce to allow them.		 09:26
patrickb Just to get the word out and hopefully some people smiling: The last two zef updates bring in optimizations that can speed up module installation by more than factor 10: github.com/ugexe/zef/commit/d22354...cd1d708686 and github.com/ugexe/zef/commit/008771...de381fc450 09:48
(ugexe++)++
patrickb xliff: ^ This may be relevant for your GTK modules. 09:49
tellable6 patrickb, I'll pass your message to Xliff
tib hello :) 09:53
what happens to signals during profiling ? 09:54
timotimo there shouldn't be anything special about signals during profiling
tib could you please test this on your side ? 09:58
gist.github.com/thibaultduponchell...81dd51cabe
(or anybody)
you have to ctrl-c quickly (one second or so) 09:59
if I wait some second, signal seems not caught at all :/ 10:00
sometimes* 10:01
tyil ooh, zef speedups are very welcome :D
tib zef speedups ? 10:03
tyil tib: zef, the module installer, has gotten a couple updates to (vastly) improve module installation, it looks like 10:04
tib ah ok :) 10:06
moritz tyil++ 10:13
lizmat vrurg++ # www.reddit.com/r/rakulang/comments...;context=3 10:15
www.reddit.com/r/rakulang/comments...s/gckupey/ # better link 10:16
tib One more and I stop bothering you 10:34
signal(SIGINT).tap( { say "Thank you for your attention"; exit 0 } ); for (1..200_000_000) { } 10:35
when profiled, give me 90% of time spent in GC 10:36
but curiously it takes ~14sec to run with profiler against 1.5sec without so it could comes from profiling 10:38
lizmat tib: yeah, Heisenberg interfering there
the signal() starts the supervisor thread 10:40
that's burning some CPU
tib ah ok 10:41
lizmat but basically the issue with the profiler is that it really doesn't measure CPU usage, but wallclock
time spent in GC is basically the wallclock time not spent in code 10:42
tib Is there a way to stop a running raku process and get a profiler report
?
lizmat with the debug server you should be able to to that, afaik, but I've never done that myself
Comma uses that though, so maybe that's the easiest way to check that out 10:43
tib hmm good idea
On my computer the for loop : for (1..3_000_000_000) { } is so crazy that I can't get the end of the profiling run even after minutes whereas or (1..2_000_000_000) { } takes 16secs 10:45
lizmat well, the profiler in that case, is just profiling noise basically 10:47
as the for 1..whatever loop is actually highly optimized
tib yes you're right 10:48
timotimo tib: it'll be important to know that the html profiler doesn't handle progams with multiple threads very well, which is what tapping a signal supply will cause a program to become 10:52
oh, and using "exit" makes the profiler very unhappy and i'm not entirely sure why 10:53
tib timotimo : I'm not using threads, was raku using thread for me under the hood for this for loop ? 10:55
thank you for exit tips
timotimo it will start a thread to handle signals and other asynchronous things 10:56
lizmat clickbaits rakudoweekly.blog/2020/11/16/2020-...n-renewed/ 12:30
lizmat also clickbaits vrurg.github.io/2020/11/16/Report-...-Coercions 12:31
moritz retweets 12:46
cpan-raku New module released to CPAN! Dist::Helper (1.0.1) by 03TYIL 12:52
tib actually there is a huge performance penalty when the max range becomes too big (2**31 -1) 15:02
tib for (1..2147483646) { } 15:02
16 sec
versus
for (1..2147483647) { }
17min
timotimo Int vs int pobably 15:04
well, more like Int with a big integer in it vs Int with a small big integer in it
tib yes, very big penalty anyway 15:05
timotimo you could profile with a c-level profile such as perf to see whee that time is spnet 15:06
tib "c-level profile" ?
ah yes, another profiler you mean 15:07
timotimo a profiler you would run against any kind of program
valgrind's callgrind and cachegrind, perf, these ae my top 3 used 15:08
the version with the number one higher has not only a boatload more jit frames that show up in the "above 0.5%" pat, but also has GC stuff dominating the top spots 15:12
holyghost .tell Xliff my macbook is back online, no need for the keys, thanks 17:17
tellable6 holyghost, I'll pass your message to Xliff
JJAtria[m] Oh, nice. Faster zef for the win 🏃 17:23
holyghost is away since 05:31:05 - automagically set away after 10 min idling. Msglog [ON] 17:27
lizmat holyghost: please switch off your automatic idling message 17:30
[Coke] if not, we can boot 'em. 17:32
holyghost is back 17:34
I cannot
perryprog Why not?
holyghost is away since 05:48:03 - automagically set away after 10 min idling. Msglog [ON] 17:44
timotimo but seriously, though ... please turn it off 17:48
Grinnz it's impolite to use auto-away messages on IRC 17:55
timotimo still the tiniest bit interested in building that simulator ... 17:58
tyil apart from his client being from 2002, I do wonder why he would be unable to remove that 17:59
timotimo worst case, write a proxy in raku that tosses the lines out 18:05
holyghost is back 18:17
fixified
perryprog :D
holyghost I hope :-)
reminds me of BitchX days
auto away set to 0 so it should work 18:18
or not :-)
perryprog In before every time you are idle for a second a message is mass-sent
holyghost indeed
non terminal IRC is somewhat off these days, I am not used to that 18:20
anyway back to config file writing and Makefiles :-) 18:21
AlexSnowman wassaaaap!! ٩(̾●̮̮̃̾•̃̾)۶٩(̾●̮̮̃̾•̃̾)۶٩(̾●̮̮̃̾•̃̾)۶٩(̾●̮̮̃̾•̃̾)۶٩(̾●̮̮̃̾•̃̾)۶٩(̾●̮̮̃̾•̃̾)۶ 18:30
Geth advent: 1cc8ce1fa9 | p6steve++ (committed using GitHub Web editor) | raku-advent-2020/authors.md
Update authors.md
 19:27
Geth advent: 107f4e6b74 | (Elizabeth Mattijsen)++ | raku-advent-2020/authors.md
Add lizmat as author
 20:06
Geth doc: 9ba21c39de | (Stoned Elipot)++ | doc/Type/Block.pod6
Use code formatting and xref type
 21:41
linkable6 Link: docs.raku.org/type/Block
tib timotimo : does moarperf handle correctly threads runs ? (I have it also) Is there a way to install signal without adding a new thread ? 22:26
Geth doc: 5d7bf64626 | (Stoned Elipot)++ | doc/Type/Slip.pod6
xref operator
 22:27
timotimo yes, no
linkable6 Link: docs.raku.org/type/Slip
[Coke] who manages Planet Raku? (wondering if we need to update the feed from Jo to be raku tagged articles only) 22:28
timotimo hm. maybe you can do it by going for the low-level functions with NativeCall
tib Ok thank you :) 22:31
rba [Coke]: I try to keep planet.raku.org/ alive. The feeds are configured in github.com/Raku/planet.raku.org/bl...perlanetrc 22:34
[Coke] rba: thanks. Quick look at the rss feed I don't see an easy way to restrict it 22:36
no tags in the feed.
rba [Coke]: Quick look too and not find too...
[Coke] ah well. Thanks for the quick response. 22:37
lizmat [Coke] I think we need to rethink planet.raku.org to be a collector of articles, rather than of feeds
[Coke] if we do that, it requires much more maintenance. 22:39
(which is fine, just more work)
[Coke] ~~ 22:40
JJAtria[m] I got a bug report against HTTP::Tiny that looks like it might be a problem in OpenSSL. It's a bit too low-level for me, though. Does anybody have any ideas? gitlab.com/jjatria/http-tiny/-/iss..._449518826
Geth doc: 0c6e715e03 | (Stoned Elipot)++ | doc/Type/Thread.pod6
Fix link and use code formatting
 22:43
linkable6 Link: docs.raku.org/type/Thread

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!