🦋 Welcome to the MAIN() IRC channel of the Raku Programming Language (raku.org). Log available at irclogs.raku.org/raku/live.html . If you're a beginner, you can also check out the #raku-beginner channel!
Set by lizmat on 6 September 2022.
guifa_ rf: honestly using JSON isn't a terrible option 00:16
rf guifa: No sorry, I mean I'm taking in JSON then doing stuff to it, then ideally I need to return a Hash 01:37
rf raiph: I think you lead me down the right route with that comment. I am going to play around with this and report back. Sorry if my question was poorly worded. 01:43
Basically, if anyone is wondering, I'm writing a hyper-fast json decoder with C++ and binding it to Raku 01:45
MasterDuke using lemire's simdjson? 01:53
rf That's the plan. 01:54
MasterDuke nice
rf .tell tonyo I broke the resource command using the emacs shell, not sure, but it appended a new line to the resource value 02:25
tellable6 rf, I'll pass your message to tonyo
rf Goodnight folks 03:06
guifa_ .tell rf Ah, now I see what you're trying to do ^_^ I mean, it's kind of a cool idea to think about returning Raku stuff directly from C. 03:42
tellable6 guifa_, I'll pass your message to rf
rf Morning folks. 13:58
tellable6 2023-02-12T03:42:17Z #raku <guifa_> rf Ah, now I see what you're trying to do ^_^ I mean, it's kind of a cool idea to think about returning Raku stuff directly from C.
rf guifa_: Yeah, a lot of languages with FFI let you do this. Maybe we can return a MoarVMHash (MVMHash) but I'm not sure. 14:02
p6steve this is a cool idea ... I put a non-answer on SO already ... but now I understand the goal, it sounds a little like what I did here in Dan::Polars (line 151 ish) github.com/p6steve/raku-Dan-Polars...rs.rakumod 14:26
so my thing is to get a Rust Polars DataFrame column (e.g. of str) over to raku .... on the Rust side I make a big Array of Str and then on Raku side I go my $array := CArray[uint8].allocate($chars); Buf.new($array.list).decode.split('","'); 14:29
turns out that the Raku .list & .decode are comparatively very slow ... in my case it is something like 1sec to read a csv on the Rust side and then 10sec to do the decode 14:31
sooo... you could consider storing the HashMap on the C++ side and exposing it as a function within raku ... you can use e.g. @lizmats raku.land/zef:lizmat/Hash::Agnostic as a template for that 14:34
guifa_ rf: my guess is that the design of Rakudo makes that a bit trickier, because MoarVM isn't supposed to know what Rakudo's actually doing structure wise, and if Rakudo changes how it deals with a hash, for instance, then the code would need to be updated / recompiled accordingly 14:36
(I hope I'm not coming off saying don't try it, I would LOVE to be able to get a crazy fast JSON and XML parser using the technique) 14:37
rf This is why I think I'm ultimately going to just do callbacks using a naming scheme
I think I can achieve super fast results still.
p6steve raku lazy model is great for this kind of thing 14:38
rf Yeah actually I think this method using callbacks will be faster overall than encoding to a raku object. I think I can get to the 900mb/s JSON decoding goal like this 14:41
Found a new bug in Distribution::Builder:Cmake, so going to fix this first. 14:42
Anton Antonov I have a design question: What is a good module name for time series / temporal data manipulation? Here are names I consider: "Data::TimeSeries", "Data::Temporal", "TemporalData". Any suggestions would be appreciated! 15:21
tonyo . 15:22
tellable6 2023-02-11T16:43:37Z #raku <rf> tonyo I may be missing something but I think fez init should take a name, and make the module inside of the directory rather than within the CWD
2023-02-11T16:46:15Z #raku <rf> tonyo Besides that everything seems to be working fairly well, not exactly sure where cmd sources its commands from though.
2023-02-11T18:28:35Z #raku <rf> tonyo I also think the name field of META6 should be: Module::Bob instead of Module-Bob
2023-02-12T02:25:40Z #raku <rf> tonyo I broke the resource command using the emacs shell, not sure, but it appended a new line to the resource value
tonyo rf: good feed back - the `name` in META6 should be a dist name and not the main module name, though. it's something we've been trying to get to happen but in practice people (including myself) just use the module name, with the tool i think it makes sense to make the default what is the best practice 15:23
rf: you can also pass the module name like `fez init <some module>`, if you omit the module name then it'll prompt 15:25
rf Yeah, I just find it weird that it makes it in CWD
But that's just me, curious what others feel like
tonyo you're probably right 15:29
i'm updating it now
rf Cool, let me know and I can test it :D
guifa_ I think the most frustrating thing right now working with RakuAST is that the errors are very LTA 15:31
guifa_ is not complaining to be clear! Getting stuff working is step one 15:32
rf ugexe: Hey if I have a submodule in my dist will zef pull it down when I install?
If not I can just bundle it with my dist, but would be nice to keep bundle size down for zef, feel bad pushing a 25mb submodule lol 15:33
guifa_ rf: yes, it will
rf Sweet!
guifa_ As long as you've declared it as a dependency
rf No I mean like git submodule sorry. 15:34
Should have clarified.
guifa_ oh that I don't know then
rf In this case simdjson
tonyo rf: if you pull on the dist branch you should see a fix for it to mk the dir and not init in the same dir 15:37
rf tonyo++ I'll do that 15:38
Holy crap, simdjson bindings working lets goo
Gonna have to benchmark this sometime today
tonyo re: resources, how do you mean it added a new line? 15:39
rf I ran resources in emacs shell: fez resource "libraries/blahblah" 15:40
and it appended: "libraries/blahlbah\n"
tonyo ah yea, that command is used to add a resource to your project 15:41
rf Yeah but the \n was in the resource 15:41
tonyo so `fez resource xyz` will touch `resource/xyz`
rf and in the meta6.json
tonyo ahh 15:42
rf the resource created was library/blagblah\n and meta6 had it as well
Seems like a weird one haha
tonyo oh weird 15:43
tonyo okay, pushed something that _should_ fix it 15:48
rf tonyo: looks good! 15:51
tonyo flight to ecuador may be enough time for me to get rid of the system dependency of gzip/tar headache i currently have 16:06
likely on the zef side too 16:07
rf: regarding commands, you can add a file `.zef` with `{"commands": {"some-command": ["echo", "hello world!"] } }` and run `fz r some-command` then it should echo `hello world!` 16:10
we've gone back and forth a ton on whether that stuff should live in META or in a config file, i think we want the META as slim as possible (since it's read frequently) 16:11
rf Yeah that makes sense 16:14
ugexe No, including a sub module won’t do anything re: zef 16:23
release it as a separate distribution and depend on that 16:24
rf I had to do some hideousness to bind simdjson but I think it works! 17:27
rf Here comes the test folks 18:49
Json::Simd vs. Json::Tiny
Or is Json::Fast the new most popular? 18:50
Anton Antonov Good question. I mostly use "JSON::Fast" because I think it is most popular and/or maintained. (After looking through raku.land, etc.) 18:56
rf Well simdjson is on-par with JSON::Fast, and leagues ahead of JSON::Tiny 18:58
I think that if I can optimize the way I handle treed data it might be faster
Nemokosch JSON::Fast is really mostly written in NQP 19:14
rf Yeah i'm converting the SIMD code to NQP 19:17
Gonna see if I can beat it
p6steve hi Anton - I like that all your modules are Data::Xxxx - that helps me to point consistently at your stuff (and maybe other "exploded" Data related modules from other authors) when needed 19:37
in this case, I personally prefer Data::TimeSeries
Anton Antonov @p6steve Thank you for your feedback! I was considering making a module title "TemporalData", but now I plan to rethink that... 19:47
Nemokosch @Anton Antonov don't depend on HTML::Escape, pretty please 19:49
that module is so dead
even copying the logic makes more sense than depending on it
I tend to replace it with HTML::Entity::Fast, for that matter 19:50
rf 2.3% faster than JSON::Fast lol 19:55
That's one test, about 180mb of json
tonyo: Another weird thing I found, the test it generates appends 2 spaces to the left of everything for some reason 19:56
.tell tonyo Another weird thing I found, the test it generates appends 2 spaces to the left of everything for some reaso
tellable6 rf, I'll pass your message to tonyo
Anton Antonov Thanks! Really, good to know. I will change that dependency soon. (Today/tonight.) 19:57
rf .tell tonyo Also, test files should be .rakutest not .t anymore iircs 19:59
tellable6 rf, I'll pass your message to tonyo
Nemokosch these thoughts really circulate a lot like memes 20:01
it wasn't at all clear whether .t will be deprecated, for a long long time
in the "path to Raku" issue, that change was not proposed 20:02
and all of a sudden, in the last couple of months, this topic became hot for whatever reason
... and this really is just a very very banal and irrelevant case of "the community should communicate much more" 20:03
Geth doc/main: a8009b7a89 | (Will Coleda)++ | 2 files
Remove AUTHORS section

Only used on these two pages, not consistent with overall styling, which pushes CREDITS more than per-page notes.
20:15
Anton Antonov I just tried that. Unfortunately, with "HTML::Entity::Fast" => are converted into =⃥> / &bne;&gt; . There are other problems too... 21:01
Nemokosch &bne;&gt; - isn't this correct though? 21:02
Anton Antonov This is what had before. 21:05
cdn.discordapp.com/attachments/633...Escape.png
This is what I have with "HTML:::Entity::Fast". 21:06
cdn.discordapp.com/attachments/633...y-Fast.png
Nemokosch regarding the newlines, this kinda seems like a misuse of "html escaping" 21:09
it's double escaped
pingu how can I tokenize space separated parts of a string but also keep the tokens as lines 21:12
pingu so ("a b c def\n"g h i jkl\n") would become 'a','b','c','def' <new arraydimention> 'g','h','i','jkl' 21:13
that would be a [2][4] array 21:14
if I removed the \n
gfldex m: "a b c def\n"g h i jkl\n".lines».split(/ /).say; 21:15
camelia ===SORRY!=== Error while compiling <tmp>
Two terms in a row
at <tmp>:1
------> "a b c def\n"⏏g h i jkl\n".lines».split(/ /).say;
expecting any of:
infix
infix stopper
statement end
st…
pingu im not too bothered if the newlines are gone
gfldex m: "a b c def\ng h i jkl\n".lines».split(/ /).say;
camelia ===SORRY!=== Error while compiling <tmp>
Null regex not allowed
at <tmp>:1
------> a b c def\ng h i jkl\n".lines».split(/ /⏏).say;
gfldex m: "a b c def\ng h i jkl\n".lines».split(' ').say;
camelia ((a b c def) (g h i jkl))
rf m: "a b c def\ng h i jkl\n".line>>.comb>>.grep(* != ' ').say; 21:22
camelia No such method 'line' for invocant of type 'Str'. Did you mean any of
these: 'lines', 'Int', 'List', 'UInt', 'clone', 'list', 'min', 'none',
'one', 'sin', 'sinh', 'sink', 'slice'?
in block <unit> at <tmp> line 1
rf m: "a b c def\ng h i jkl\n".lines>>.comb>>.grep(* != ' ').say;
camelia Cannot convert string to number: base-10 number must begin with valid digits or '.' in '⏏a' (indicated by ⏏)
in block <unit> at <tmp> line 1
rf m: "a b c def\ng h i jkl\n".lines>>.comb>>.grep(* ne ' ').say;
camelia ((a b c d e f) (g h i j k l))
guifa_ does a dance 21:24
I just got number formatting to work with RakuAST to a minimally functional level
just have to handle decimal digits which shouldn't take too long now 21:25
rf Nice! 21:26
pingu would this work on a file too? my $fp = open "data.ldr", :r;
my @data_ldr = $fp.IO.lines.map(*.comb.List);
close($fp);
gfldex the `close` wont work but is redundent anyway 21:28
and you will need either `.slurp` or `.open` 21:29
rf Well, here's the fruit of my last 2 days 21:31
github.com/rawleyfowler/JSON-Simd
thundergnat m: dd my @array = "a b c def\ng h i jkl\n".lines».words».Array; say @array[1;3]
camelia Array @array = [["a", "b", "c", "def"], ["g", "h", "i", "jkl"]]
jkl
pingu thx thats just what I needed 21:34
thundergnat m: dd my @array = "a b c def\ng h i jkl\nm n o pqr\n".lines».words».Array; say @array[*;2] 21:35
camelia Array @array = [["a", "b", "c", "def"], ["g", "h", "i", "jkl"], ["m", "n", "o", "pqr"]]
(c i o)
thundergnat :-)
pingu if (@data_ldr[$line][15].contains($partID)) 21:53
no such method .contains for 'any'
Nemokosch have you considered that you are over-indexing? 21:56
pingu theres 3 lines with 15 tokens 21:57
0 to 14
so I did < 15 21:58
pastebin.com/krDrDYuh
ah wait ok 21:59
yeah [14]
pingu and I think @elems is not doing per line at all 22:06
pingu yes because it isnt letting me index individual tokens 22:18
pingu oh wait it is 22:19
aight got it 22:25