🦋 Welcome to the MAIN() IRC channel of the Raku Programming Language (raku.org). Log available at irclogs.raku.org/raku/live.html . If you're a beginner, you can also check out the #raku-beginner channel!
Set by lizmat on 6 September 2022.
00:00 reportable6 left 00:03 reportable6 joined 00:04 jpn joined, Maylay_ left 00:13 sena_kun left, jpn left
guifa_ rf: honestly using JSON isn't a terrible option 00:16
00:16 Maylay joined 00:22 Maylay left 00:27 Maylay joined 00:31 cfa left 00:40 jpn joined 00:45 jpn left 01:10 Manifest0 left 01:35 jpn joined
rf guifa: No sorry, I mean I'm taking in JSON then doing stuff to it, then ideally I need to return a Hash 01:37
01:42 jpn left
rf raiph: I think you lead me down the right route with that comment. I am going to play around with this and report back. Sorry if my question was poorly worded. 01:43
Basically, if anyone is wondering, I'm writing a hyper-fast json decoder with C++ and binding it to Raku 01:45
MasterDuke using lemire's simdjson? 01:53
rf That's the plan. 01:54
MasterDuke nice
02:23 Maylay left 02:24 Maylay joined
rf .tell tonyo I broke the resource command using the emacs shell, not sure, but it appended a new line to the resource value 02:25
tellable6 rf, I'll pass your message to tonyo
02:28 jpn joined 02:33 jpn left 02:38 pingu left 02:49 bdju left 02:50 bdju joined, Maylay left, Maylay joined 03:00 freeside joined
rf Goodnight folks 03:06
03:06 rf left, freeside left 03:13 freeside joined 03:18 freeside left 03:23 jpn joined 03:30 jpn left 03:41 freeside joined
guifa_ .tell rf Ah, now I see what you're trying to do ^_^ I mean, it's kind of a cool idea to think about returning Raku stuff directly from C. 03:42
tellable6 guifa_, I'll pass your message to rf
03:45 freeside left 04:17 jpn joined 04:22 jpn left 04:44 freeside joined 04:48 freeside left 04:50 systems joined 05:11 jpn joined 05:18 jpn left 05:22 eponym joined 05:23 epony left, eponym left 05:37 Maylay left 05:38 Maylay joined 05:51 jpn joined 05:56 jpn left 05:57 wanderer left 06:00 reportable6 left 06:02 reportable6 joined 06:07 freeside joined 06:12 freeside left, gabiruh left 06:15 gabiruh joined 06:47 jpn joined 06:53 jpn left 07:01 jpn joined 07:07 jpn left 07:36 freeside joined 07:43 freeside left 08:09 razetime joined 08:17 systems left 08:22 freeside joined 08:26 freeside left 08:35 jpn joined 08:40 jpn left 09:40 linkable6 left, coverable6 left, tellable6 left, releasable6 left, committable6 left, reportable6 left, notable6 left, evalable6 left, nativecallable6 left, bloatable6 left, bisectable6 left, statisfiable6 left, greppable6 left, benchable6 left, unicodable6 left, quotable6 left, squashable6 left, shareable6 left, sourceable6 left, committable6 joined 09:41 bisectable6 joined, benchable6 joined, bloatable6 joined, nativecallable6 joined, greppable6 joined, unicodable6 joined, tellable6 joined, evalable6 joined, sourceable6 joined, quotable6 joined, notable6 joined 09:42 releasable6 joined, shareable6 joined, coverable6 joined, linkable6 joined, reportable6 joined 09:43 squashable6 joined, statisfiable6 joined 09:48 Sgeo left 09:49 razetime left 10:02 freeside joined 10:04 ab5tract joined 10:07 freeside left 10:08 sena_kun joined 10:24 jpn joined 10:36 freeside joined 10:38 Manifest0 joined 10:41 freeside left 10:52 jpn left 10:58 jpn joined 11:01 sena_kun left 11:02 sena_kun joined 11:54 freeside joined 11:58 freeside left 12:00 reportable6 left, reportable6 joined 12:16 frost61 joined 12:20 frost61 left 12:24 jpn left 12:28 KotH left 12:36 Maylay left 12:37 jpn joined 12:41 Maylay joined 12:44 jpn left 12:45 freeside joined 12:47 Manifest0 left 12:51 freeside left 12:52 Geth left, Geth joined 13:04 jpn joined 13:30 jpn left 13:31 freeside joined 13:35 freeside left 13:36 freeside left 13:58 rf joined
rf Morning folks. 13:58
tellable6 2023-02-12T03:42:17Z #raku <guifa_> rf Ah, now I see what you're trying to do ^_^ I mean, it's kind of a cool idea to think about returning Raku stuff directly from C.
13:59 freeside joined
rf guifa_: Yeah, a lot of languages with FFI let you do this. Maybe we can return a MoarVMHash (MVMHash) but I'm not sure. 14:02
14:08 freeside left 14:13 epony joined
p6steve this is a cool idea ... I put a non-answer on SO already ... but now I understand the goal, it sounds a little like what I did here in Dan::Polars (line 151 ish) github.com/p6steve/raku-Dan-Polars...rs.rakumod 14:26
so my thing is to get a Rust Polars DataFrame column (e.g. of str) over to raku .... on the Rust side I make a big Array of Str and then on Raku side I go my $array := CArray[uint8].allocate($chars); Buf.new($array.list).decode.split('","'); 14:29
turns out that the Raku .list & .decode are comparatively very slow ... in my case it is something like 1sec to read a csv on the Rust side and then 10sec to do the decode 14:31
14:32 freeside joined, cfa joined
sooo... you could consider storing the HashMap on the C++ side and exposing it as a function within raku ... you can use e.g. @lizmats raku.land/zef:lizmat/Hash::Agnostic as a template for that 14:34
14:36 jpn joined, freeside left
guifa_ rf: my guess is that the design of Rakudo makes that a bit trickier, because MoarVM isn't supposed to know what Rakudo's actually doing structure wise, and if Rakudo changes how it deals with a hash, for instance, then the code would need to be updated / recompiled accordingly 14:36
(I hope I'm not coming off saying don't try it, I would LOVE to be able to get a crazy fast JSON and XML parser using the technique) 14:37
rf This is why I think I'm ultimately going to just do callbacks using a naming scheme
I think I can achieve super fast results still.
p6steve raku lazy model is great for this kind of thing 14:38
rf Yeah actually I think this method using callbacks will be faster overall than encoding to a raku object. I think I can get to the 900mb/s JSON decoding goal like this 14:41
Found a new bug in Distribution::Builder:Cmake, so going to fix this first. 14:42
14:52 jpn left 15:01 freeside joined 15:05 freeside left 15:13 jpn joined 15:18 jpn left
Anton Antonov I have a design question: What is a good module name for time series / temporal data manipulation? Here are names I consider: "Data::TimeSeries", "Data::Temporal", "TemporalData". Any suggestions would be appreciated! 15:21
tonyo . 15:22
tellable6 2023-02-11T16:43:37Z #raku <rf> tonyo I may be missing something but I think fez init should take a name, and make the module inside of the directory rather than within the CWD
2023-02-11T16:46:15Z #raku <rf> tonyo Besides that everything seems to be working fairly well, not exactly sure where cmd sources its commands from though.
2023-02-11T18:28:35Z #raku <rf> tonyo I also think the name field of META6 should be: Module::Bob instead of Module-Bob
2023-02-12T02:25:40Z #raku <rf> tonyo I broke the resource command using the emacs shell, not sure, but it appended a new line to the resource value
tonyo rf: good feed back - the `name` in META6 should be a dist name and not the main module name, though. it's something we've been trying to get to happen but in practice people (including myself) just use the module name, with the tool i think it makes sense to make the default what is the best practice 15:23
rf: you can also pass the module name like `fez init <some module>`, if you omit the module name then it'll prompt 15:25
rf Yeah, I just find it weird that it makes it in CWD
But that's just me, curious what others feel like
15:28 QhpAptyj9hj0RQwM joined
tonyo you're probably right 15:29
i'm updating it now
rf Cool, let me know and I can test it :D
guifa_ I think the most frustrating thing right now working with RakuAST is that the errors are very LTA 15:31
guifa_ is not complaining to be clear! Getting stuff working is step one 15:32
rf ugexe: Hey if I have a submodule in my dist will zef pull it down when I install?
If not I can just bundle it with my dist, but would be nice to keep bundle size down for zef, feel bad pushing a 25mb submodule lol 15:33
guifa_ rf: yes, it will
rf Sweet!
guifa_ As long as you've declared it as a dependency
rf No I mean like git submodule sorry. 15:34
Should have clarified.
guifa_ oh that I don't know then
rf In this case simdjson
tonyo rf: if you pull on the dist branch you should see a fix for it to mk the dir and not init in the same dir 15:37
rf tonyo++ I'll do that 15:38
Holy crap, simdjson bindings working lets goo
Gonna have to benchmark this sometime today
tonyo re: resources, how do you mean it added a new line? 15:39
rf I ran resources in emacs shell: fez resource "libraries/blahblah" 15:40
and it appended: "libraries/blahlbah\n"
15:41 Geth left
tonyo ah yea, that command is used to add a resource to your project 15:41
15:41 Geth joined
rf Yeah but the \n was in the resource 15:41
tonyo so `fez resource xyz` will touch `resource/xyz`
rf and in the meta6.json
tonyo ahh 15:42
rf the resource created was library/blagblah\n and meta6 had it as well
Seems like a weird one haha
tonyo oh weird 15:43
15:45 Geth left, Geth joined
tonyo okay, pushed something that _should_ fix it 15:48
15:50 Sauvin left
rf tonyo: looks good! 15:51
16:01 Sauvin joined
tonyo flight to ecuador may be enough time for me to get rid of the system dependency of gzip/tar headache i currently have 16:06
likely on the zef side too 16:07
rf: regarding commands, you can add a file `.zef` with `{"commands": {"some-command": ["echo", "hello world!"] } }` and run `fz r some-command` then it should echo `hello world!` 16:10
we've gone back and forth a ton on whether that stuff should live in META or in a config file, i think we want the META as slim as possible (since it's read frequently) 16:11
rf Yeah that makes sense 16:14
16:18 razetime joined 16:21 Maylay left
ugexe No, including a sub module won’t do anything re: zef 16:23
release it as a separate distribution and depend on that 16:24
16:29 Maylay joined 16:55 freeside joined 16:59 freeside left 17:21 freeside joined 17:25 freeside left
rf I had to do some hideousness to bind simdjson but I think it works! 17:27
17:28 Manifest0 joined, cfa left 17:30 razetime left 17:52 freeside joined 17:57 freeside left 18:00 reportable6 left 18:01 Sgeo joined 18:03 reportable6 joined 18:12 freeside joined 18:16 freeside left 18:22 freeside joined 18:28 freeside left 18:38 freeside joined 18:42 freeside left
rf Here comes the test folks 18:49
Json::Simd vs. Json::Tiny
Or is Json::Fast the new most popular? 18:50
18:50 epony left
Anton Antonov Good question. I mostly use "JSON::Fast" because I think it is most popular and/or maintained. (After looking through raku.land, etc.) 18:56
rf Well simdjson is on-par with JSON::Fast, and leagues ahead of JSON::Tiny 18:58
I think that if I can optimize the way I handle treed data it might be faster
Nemokosch JSON::Fast is really mostly written in NQP 19:14
rf Yeah i'm converting the SIMD code to NQP 19:17
Gonna see if I can beat it
19:18 freeside joined 19:24 jpn joined, freeside left 19:28 jpn left
p6steve hi Anton - I like that all your modules are Data::Xxxx - that helps me to point consistently at your stuff (and maybe other "exploded" Data related modules from other authors) when needed 19:37
in this case, I personally prefer Data::TimeSeries
19:41 kybr joined
Anton Antonov @p6steve Thank you for your feedback! I was considering making a module title "TemporalData", but now I plan to rethink that... 19:47
Nemokosch @Anton Antonov don't depend on HTML::Escape, pretty please 19:49
that module is so dead
even copying the logic makes more sense than depending on it
I tend to replace it with HTML::Entity::Fast, for that matter 19:50
rf 2.3% faster than JSON::Fast lol 19:55
That's one test, about 180mb of json
tonyo: Another weird thing I found, the test it generates appends 2 spaces to the left of everything for some reason 19:56
.tell tonyo Another weird thing I found, the test it generates appends 2 spaces to the left of everything for some reaso
tellable6 rf, I'll pass your message to tonyo
Anton Antonov Thanks! Really, good to know. I will change that dependency soon. (Today/tonight.) 19:57
rf .tell tonyo Also, test files should be .rakutest not .t anymore iircs 19:59
tellable6 rf, I'll pass your message to tonyo
20:01 freeside joined
Nemokosch these thoughts really circulate a lot like memes 20:01
it wasn't at all clear whether .t will be deprecated, for a long long time
in the "path to Raku" issue, that change was not proposed 20:02
and all of a sudden, in the last couple of months, this topic became hot for whatever reason
... and this really is just a very very banal and irrelevant case of "the community should communicate much more" 20:03
20:04 NemokoschKiwi joined 20:06 freeside left 20:08 jgaz joined 20:14 grizzlysmit joined 20:15 grizzlysmit left
Geth doc/main: a8009b7a89 | (Will Coleda)++ | 2 files
Remove AUTHORS section

Only used on these two pages, not consistent with overall styling, which pushes CREDITS more than per-page notes.		 20:15
20:24 Geth left, Geth joined 20:28 Bocaneri joined, Bocaneri is now known as Guest6558 20:29 Guest6558 is now known as SenFache 20:30 Sauvin left 21:01 pingu joined
Anton Antonov I just tried that. Unfortunately, with "HTML::Entity::Fast" => are converted into =⃥> / &bne;&gt; . There are other problems too... 21:01
Nemokosch &bne;&gt; - isn't this correct though? 21:02
Anton Antonov This is what had before. 21:05
cdn.discordapp.com/attachments/633...Escape.png
This is what I have with "HTML:::Entity::Fast". 21:06
cdn.discordapp.com/attachments/633...y-Fast.png
21:08 freeside joined
Nemokosch regarding the newlines, this kinda seems like a misuse of "html escaping" 21:09
it's double escaped
pingu how can I tokenize space separated parts of a string but also keep the tokens as lines 21:12
21:12 freeside left
pingu so ("a b c def\n"g h i jkl\n") would become 'a','b','c','def' <new arraydimention> 'g','h','i','jkl' 21:13
that would be a [2][4] array 21:14
if I removed the \n
gfldex m: "a b c def\n"g h i jkl\n".lines».split(/ /).say; 21:15
camelia ===SORRY!=== Error while compiling <tmp>
Two terms in a row
at <tmp>:1
------> "a b c def\n"⏏g h i jkl\n".lines».split(/ /).say;
expecting any of:
infix
infix stopper
statement end
st…
pingu im not too bothered if the newlines are gone
gfldex m: "a b c def\ng h i jkl\n".lines».split(/ /).say;
camelia ===SORRY!=== Error while compiling <tmp>
Null regex not allowed
at <tmp>:1
------> a b c def\ng h i jkl\n".lines».split(/ /⏏).say;
gfldex m: "a b c def\ng h i jkl\n".lines».split(' ').say;
camelia ((a b c def) (g h i jkl))
rf m: "a b c def\ng h i jkl\n".line>>.comb>>.grep(* != ' ').say; 21:22
camelia No such method 'line' for invocant of type 'Str'. Did you mean any of
these: 'lines', 'Int', 'List', 'UInt', 'clone', 'list', 'min', 'none',
'one', 'sin', 'sinh', 'sink', 'slice'?
in block <unit> at <tmp> line 1
rf m: "a b c def\ng h i jkl\n".lines>>.comb>>.grep(* != ' ').say;
camelia Cannot convert string to number: base-10 number must begin with valid digits or '.' in '⏏a' (indicated by ⏏)
in block <unit> at <tmp> line 1
rf m: "a b c def\ng h i jkl\n".lines>>.comb>>.grep(* ne ' ').say;
camelia ((a b c d e f) (g h i j k l))
guifa_ does a dance 21:24
I just got number formatting to work with RakuAST to a minimally functional level
just have to handle decimal digits which shouldn't take too long now 21:25
rf Nice! 21:26
pingu would this work on a file too? my $fp = open "data.ldr", :r;
my @data_ldr = $fp.IO.lines.map(*.comb.List);
close($fp);
gfldex the `close` wont work but is redundent anyway 21:28
and you will need either `.slurp` or `.open` 21:29
21:30 jpn joined 21:31 thundergnat joined
rf Well, here's the fruit of my last 2 days 21:31
github.com/rawleyfowler/JSON-Simd
thundergnat m: dd my @array = "a b c def\ng h i jkl\n".lines».words».Array; say @array[1;3]
camelia Array @array = [["a", "b", "c", "def"], ["g", "h", "i", "jkl"]]
jkl
21:33 freeside joined
pingu thx thats just what I needed 21:34
thundergnat m: dd my @array = "a b c def\ng h i jkl\nm n o pqr\n".lines».words».Array; say @array[*;2] 21:35
camelia Array @array = [["a", "b", "c", "def"], ["g", "h", "i", "jkl"], ["m", "n", "o", "pqr"]]
(c i o)
thundergnat :-)
21:37 freeside left
pingu if (@data_ldr[$line][15].contains($partID)) 21:53
no such method .contains for 'any'
21:56 jpn left
Nemokosch have you considered that you are over-indexing? 21:56
pingu theres 3 lines with 15 tokens 21:57
0 to 14
so I did < 15 21:58
pastebin.com/krDrDYuh
ah wait ok 21:59
yeah [14]
22:06 freeside joined
pingu and I think @elems is not doing per line at all 22:06
22:12 freeside left 22:17 SenFache left
pingu yes because it isnt letting me index individual tokens 22:18
22:19 SenFache joined
pingu oh wait it is 22:19
aight got it 22:25
22:27 hythm left 22:40 freeside joined 22:45 freeside left 22:59 derpydoo joined 23:05 SenFache left 23:10 SenFache joined 23:11 sena_kun left 23:16 SenFache left 23:17 SenFache joined, QhpAptyj9hj0RQwM left 23:23 SenFache left 23:26 SenFache joined 23:32 QhpAptyj9hj0RQwM joined 23:34 derpydoo left 23:38 SenFache left 23:47 Sauvin joined 23:49 freeside joined 23:53 freeside left 23:55 QhpAptyj9hj0RQwM left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!