Perl 6 language and compiler development | Logs at colabti.org/irclogger/irclogger_log/perl6-dev | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
Set by Zoffix on 27 July 2018.
00:18 phkd6 joined, phkd6 left
AlexDaniel :) gist.github.com/AlexDaniel/dc18791...402dc34cbd 00:41
00:42 ggoebel_ left
AlexDaniel that was for HEAD^^ or something like that, not an issue anymore 00:43
00:50 dct left 01:09 Kaypie left 01:17 Drawde23 joined 01:18 Drawde23 left 01:22 lizmat left 01:28 jaagr3 joined, jaagr3 left 01:38 ggoebel_ joined, p6bannerbot sets mode: +v ggoebel_
samcv AlexDaniel: ok :) cool. will work on that releasing that tomorrow 01:51
02:12 ZzZombo left 02:23 nibbo_ joined 02:25 nibbo_ left 02:53 ZzZombo joined, p6bannerbot sets mode: +v ZzZombo 03:29 MasterDuke left 04:22 ggoebel_ left 04:50 Ven` joined 04:51 p6bannerbot sets mode: +v Ven` 04:55 Ven` left 06:19 patrickb joined, p6bannerbot sets mode: +v patrickb 06:27 robertle joined 06:28 p6bannerbot sets mode: +v robertle 06:40 ggoebel left 06:41 robertle left 06:53 ggoebel joined 06:54 p6bannerbot sets mode: +v ggoebel 07:32 illuminated12 joined 07:33 illuminated12 left, brrt joined 07:34 p6bannerbot sets mode: +v brrt 07:39 ZzZombo_ joined, p6bannerbot sets mode: +v ZzZombo_ 07:41 ZzZombo left, ZzZombo__ joined, ZzZombo__ is now known as ZzZombo, p6bannerbot sets mode: +v ZzZombo 07:44 ZzZombo_ left 07:50 Ven` joined, masak joined 07:51 p6bannerbot sets mode: +v Ven`, p6bannerbot sets mode: +v masak 07:53 lizmat joined 07:54 dogbert17 left, p6bannerbot sets mode: +v lizmat
lizmat Files=1253, Tests=76140, 338 wallclock secs (15.48 usr 5.60 sys + 2380.69 cusr 228.51 csys = 2630.28 CPU) 07:54
08:00 awwaiid left 08:14 exferenceBot28 joined 08:19 exferenceBot28 left 08:42 brrt left 08:51 dogbert2 joined 08:52 p6bannerbot sets mode: +v dogbert2 08:56 brrt joined, p6bannerbot sets mode: +v brrt 09:00 haxxpop28 joined 09:04 haxxpop28 left
samcv [Tux]: you around? 09:05
09:15 [TuxCM] joined 09:16 p6bannerbot sets mode: +v [TuxCM] 09:54 Ven` left 09:55 ZzZombo left 09:56 Ven` joined 09:57 p6bannerbot sets mode: +v Ven` 10:16 brrt left 10:21 brrt joined, p6bannerbot sets mode: +v brrt
Geth nqp/nqp-mbc: 21 commits pushed by (Stefan Seifert)++
review: github.com/perl6/nqp/compare/a32e1...b17eea2cc0		 10:43
11:03 Ven` left 11:15 [TuxCM] left 11:19 [Tux] joined, [Tux] is now known as [TuxCM] 11:20 p6bannerbot sets mode: +v [TuxCM] 11:25 innovati joined 11:27 ZzZombo joined, p6bannerbot sets mode: +v ZzZombo 11:30 innovati left
[TuxCM] I was running a timing when my PC went out. I hope the two are not related 11:31
luckily my daughter was at home to reboot the box
11:38 Ven` joined 11:39 p6bannerbot sets mode: +v Ven` 11:41 Kaiepi joined 11:42 p6bannerbot sets mode: +v Kaiepi 11:51 brrt left
Geth rakudo/postrelease-opts: 5f55d76272 | (Jonathan Worthington)++ | src/Perl6/World.nqp
Use a simple atkey in generated BUILDALL

We're already using the existskey op, so may as well go the whole hog and have simpler, faster code here.		 11:54
lizmat jnthn: good catch! 11:59
maybe I should port that to the 2018.09 release ?
jnthn No, not worth the risk 12:00
And it's a win but not a massive one since the costs in construction are elsewhere (I'm looking into that) 12:01
lizmat indeed, I don't see any difference in test-t performance: guess the AT-KEY was very quickly inlined already ? 12:05
jnthn Yeah, it was these days
I did see a difference in a construction benchmark 12:06
But small
12:15 ggoebel_ joined 12:16 p6bannerbot sets mode: +v ggoebel_ 12:48 brrt joined, p6bannerbot sets mode: +v brrt 12:52 Guest52089 joined 12:54 Guest52089 left
dogbert2 m: .contains: "meow" for ^10000_00; say now - INIT now 12:54
camelia 7.27520441
AlexDaniel 6c: .contains: "meow" for ^5000_00; say now - INIT now 12:57
committable6 AlexDaniel, gist.github.com/1d453ef107d4ce5a98...d861cf4e82 12:58
dogbert2 AlexDaniel: RT #132280 12:59
synopsebot RT#132280 [new]: rt.perl.org/Ticket/Display.html?id=132280 [REGRESSION][PERF] Cool methods that take and pass a Capture are uber slower (.contains: 'meow' for ^10000_00)
AlexDaniel I have to confess that I remember none of my tickets :) 13:00
dogbert2 btw, the RPi fix, alignment related, which went in a few days ago solved the build problems I had
AlexDaniel dogbert2: good! Thanks
lizmat m: my %h = <a b c> # throws 13:19
camelia Odd number of elements found where hash initializer expected:
Found 3 (implicit) elements:
Last element seen: "c"
in block <unit> at <tmp> line 1
lizmat m: my %h; %h.push( <a b c> ); dd %h # just warns, that seems inconsistent to me 13:20
camelia Trailing item in Hash.push
in block <unit> at <tmp> line 1
Hash %h = {:a("b")}
lizmat m: my %h = <a b c>; dd %h # doublechecking it throws
camelia Odd number of elements found where hash initializer expected:
Found 3 (implicit) elements:
Last element seen: "c"
in block <unit> at <tmp> line 1
13:30 patrickb left 13:36 [TuxCM] left
Geth nqp/nqp-mbc: bbdb894fff | (Stefan Seifert)++ | src/vm/moar/QAST/QASTCompilerMAST.nqp
Fix write_uint32_at only writing to the first 2^16 bytes
 14:03
14:03 AsciiWolf10 joined
diakopter oops lol 14:04
wait, what's nqp-mbc 14:05
nine diakopter: a branch where I replace MoarVM's MAST compiler with an NQP implementation with the goal of merging that with QASTCompilerMAST to get rid of MAST (or the most expensive parts at least) 14:07
diakopter oh neat. is that a significant bottleneck/stage? 14:08
14:08 AsciiWolf10 left
nine Creating the MAST causes lots of allocations 14:09
lizmat diakopter: I understand it has the potential of halving the MAST stage in time 14:21
+ needing a lot less memory, so easier on the RPi's
14:35 ggoebel_ left 14:55 [Tux] joined, p6bannerbot sets mode: +v [Tux] 14:58 [Tux] left
AlexDaniel yeah, canary is green, toaster looks alright, CI is green, releasable is happy 14:58
brrt \o/ 14:59
14:59 [Tux] joined, p6bannerbot sets mode: +v [Tux] 15:00 radeek13 joined 15:03 radeek13 left, drawkula0 joined 15:08 drawkula0 left 15:12 [TuxCM] joined, p6bannerbot sets mode: +v [TuxCM] 15:16 ggoebel_ joined 15:17 p6bannerbot sets mode: +v ggoebel_
Geth rakudo: taboege++ created pull request #2291:
Check for definedness of callframe.code when creating error message		 15:20
[Tux] waves 15:22
timotimo in *theory* we can quickly reject .contains of strings that don't look numerical when called on .Int 15:25
where did we get the "halving mast stage in time" number? 15:26
just that we're now writing mast bytecode immediately instead of first building MAST nodes?
i'd expect that only halve in the very best case, which is that qast -> MAST and MAST -> mbc both take exactly the same amount of time 15:27
but the MAST -> mbc compiler is written in C and might be a bunch faster than the QAST -> MAST part
samcv hey [Tux]! 15:28
any thoughts on whether you think "foo".encode('utf16') should add a BOM? i feel it should at least add it when we write to an actual file (start writing at position 0). but otherwise i don't know if .encode should add a BOM for utf16. here's what i said in #moarvm colabti.org/irclogger/irclogger_log...-09-19#l69 15:31
[Tux] "foo" being a file or a string? If string, no
samcv yeah just foo is a string 15:32
m: "foo".encode('utf16').say
camelia utf16:0x<66 6f 6f>
samcv [Tux]: what about to a file?
[Tux] yes
:)
samcv ok. that's what i was thinking of doing
[Tux] 👍🏻 15:33
samcv [Tux]: ok and so utf16le and utf16be as of yesterday in moarvm master will not handle BOM in files specially 15:34
like how perl 5 does (from my testing) and what the standard says. though many ignore that and treat it as a BOM (i.e. remove it from the file on read). do you think that is the right approach too? 15:35
timotimo that ... is a 3 byte big utf16 string?!
oh
no, that's three 16bit numbers
sorry
samcv so reading "\x[FEFF]hello" from a file using 'utf16' encoding will pass through "hello" while reading it with "utf16le" will pass through "\x[FEFF]hello" 15:37
[Tux] Rakudo version 2018.08-114-g19edeafd1 - MoarVM version 2018.08-92-g3e94a68f6
csv-ip5xs0.898 - 0.923
csv-ip5xs-207.673 - 7.704
csv-parser24.215 - 25.013
csv-test-xs-200.422 - 0.435
test8.655 - 9.147
test-t2.119 - 2.188
test-t --race0.902 - 0.910
test-t-2036.757 - 38.287
test-t-20 --race11.940 - 12.554
 15:38
samcv also the question is. if we do .decode('utf16') on "\x[FEFF]hello" do we get "\c[FEFF]hello"? or "hello"?
timotimo wow, quite a bounce-back 15:39
[Tux] I have no answer to that last question. I'm open to arguments for either
samcv if we don't encode the BOM except for *files* do we treat the BOM specially for .decode or do the same we do for encode (only in an actual file)
ok
same :P
[Tux] timotimo, not really I guess. I rebooted after a crash and am working on the desktop, so it is maybe a bit more busy 15:40
timotimo OK
i suppose we just hope for the data to be good on average ;)
the error can hardly be negative ;)
15:43 inked-19 joined 15:48 inked-19 left 15:56 brrt left 15:57 lizmat left
Geth rakudo: f359fa9a26 | (Tobias Boege)++ | src/core/metaops.pm6
Check for definedness of callframe.code when creating error message

This fixes #2222. The "non-dwimmy hyperop" hint uses callframe(…).code.name to detect if it recurses. In some cases callframe.code is Mu (unknown file) which leads to an error looking up .name. Inserting `andthen` solves this.		 16:12
synopsebot RAKUDO#2222 [closed]: github.com/rakudo/rakudo/issues/2222 LTA error message with non-dwimmy hyper infix
rakudo: e889d7ed68 | (Zoffix Znet)++ (committed using GitHub Web editor) | src/core/metaops.pm6
Merge pull request #2291 from taboege/callframe-code-name-in-non-dwimmy-hyperop-message

Check for definedness of callframe.code when creating error message
16:14 Ven` left 16:15 AlexDaniel left 16:17 Ven` joined 16:18 p6bannerbot sets mode: +v Ven` 16:19 AlexDaniel joined, p6bannerbot sets mode: +v AlexDaniel, Zoffix joined, p6bannerbot sets mode: +v Zoffix
Zoffix my MAST stage for core is 18s. Halving it would be a nice win :) 16:20
Geth nqp: ee16e6c4ed | (Zoffix Znet)++ | tools/build/MOAR_REVISION
[MoarVM Bump] Brings 6 commits

MoarVM bump brought: github.com/MoarVM/MoarVM/compare/2...2-g3e94a68 3e94a68 Pass through BOM with utf16le and utf16be 24860d4 let spesh discover the type of hllbool[for] 025b3cd Fix fprintf compiler warning in main.c ff75050 Make sure MVM_string_utf16le/be_decodestream are predeclared ad12d8e Fix some bugs with utf16* decodestream get_chars, and retain state 792cdd5 Add support for utf16le and utf16be decodestream		 16:22
¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...2-g3e94a68
rakudo: d275ea09bb | (Zoffix Znet)++ | tools/build/NQP_REVISION
[NQP Bump] ee16e6c [MoarVM Bump] Brings 6 commits

NQP bump brought: github.com/perl6/nqp/compare/2018....6-gee16e6c
MoarVM bump brought: github.com/MoarVM/MoarVM/compare/2...2-g3e94a68 3e94a68 Pass through BOM with utf16le and utf16be 24860d4 let spesh discover the type of hllbool[for] 025b3cd Fix fprintf compiler warning in main.c ff75050 Make sure MVM_string_utf16le/be_decodestream are predeclared ad12d8e Fix some bugs with utf16* decodestream get_chars, and retain state 792cdd5 Add support for utf16le and utf16be decodestream
¦ rakudo: version bump brought these changes: github.com/perl6/nqp/compare/2018....6-gee16e6c
Zoffix ZOFVM: Files=1309, Tests=153183, 166 wallclock secs (26.66 usr 3.44 sys + 3587.26 cusr 162.45 csys = 3779.81 CPU)
16:23 Zoffix left
Geth roast: e55749a9f6 | (Zoffix Znet)++ | MISC/bug-coverage.t
Cover hyper DWIM error

Closes github.com/rakudo/rakudo/issues/2222 R#2222 Rakudo fix: github.com/rakudo/rakudo/pull/2291/files		 16:25
synopsebot R#2222 [open]: github.com/rakudo/rakudo/issues/2222 [testneeded] LTA error message with non-dwimmy hyper infix
16:51 Ven` left 17:07 travis-ci joined, p6bannerbot sets mode: +v travis-ci
travis-ci Rakudo build failed. Zoffix Znet 'Merge pull request #2291 from taboege/callframe-code-name-in-non-dwimmy-hyperop-message 17:07
travis-ci.org/rakudo/rakudo/builds/430623589 github.com/rakudo/rakudo/compare/1...89d7ed6875
17:07 travis-ci left
buggable [travis build above] ☠ Did not recognize some failures. Check results manually. 17:07
17:27 brrt joined 17:28 p6bannerbot sets mode: +v brrt 17:35 bisectable6 left, bisectable6 joined 17:36 p6bannerbot sets mode: +v bisectable6 17:45 bisectable6 left, bisectable6 joined, ChanServ sets mode: +v bisectable6 17:46 p6bannerbot sets mode: +v bisectable6 17:50 lizmat joined 17:51 p6bannerbot sets mode: +v lizmat
AlexDaniel [Coke]: github.com/ugexe/zef/issues/272#is...-422897511 17:56
[Coke]: can you explain what the issue is exactly?
and how to resolve that without rewriting the module 17:57
17:57 brrt left 18:03 brrt joined 18:04 p6bannerbot sets mode: +v brrt 18:17 Zoffix joined, p6bannerbot sets mode: +v Zoffix
[Coke] AlexDaniel: it says "I agree with [Coke]", but doesn't say what I said. 18:19
oh.
the code that goes into rakudo is covered by the CLA.
the code that goes into "some module" isn't. 18:20
Zoffix AlexDaniel: wouldn't we be able to get more speed by making a more limited version of JSON parser? (just a guess)
AlexDaniel: like, we don't have to be able to handle custom objects, for example
AlexDaniel: also, are we talking about making a publicly-available JSON parser in core, or are we only going to improve the private one?
AlexDaniel [Coke]: so can't we just tell timotimo to contribute it to core?
[Coke] *tell*? no 18:21
*ask*? sure.
AlexDaniel ask
I didn't mean that, yes :)
[Coke] Zoffix: isn't it already publicly available?
Zoffix AlexDaniel: also, IIRC JSON::Fast creates Arrays, and it's likely more efficient to create Lists or Seqs instead (which can't be easily changed in JSON::Fast, without affecting stuff)
timotimo if all other core devs are ok with that, sure
[Coke] (as some helper routines?)
AlexDaniel Zoffix: I don't think we want to make it publicly available, just improve the internal one
Zoffix [Coke]: no, all Rakudo::Internal stuff is internal
[Coke]: I mean, you can still call it, but it'll be your fault if your code breaks. 18:22
timotimo Zoffix: i'm not sure why a Seq would be better, but a List should be doable. perhaps an IterationBuffer
Zoffix mhm
[Coke] m: dd from-json("[1]"); 18:23
camelia $[1]
[Coke] ^^
AlexDaniel not really
e: dd from-json("[1]");
evalable6 $[1]
Saw 1 occurrence of deprecated code.
================================================…
AlexDaniel, Full output: gist.github.com/dca123a1d73ffa38b7...5e4b44c73d
[Coke] aren't we talking about updating that one?
AlexDaniel yes
Zoffix [Coke]: yeah, we area
*are
[Coke]: and yeah, you're right, it doesn't have Rakudo::Internal in it's name. But there is a deprecation warning that's been in place since Christmas 18:24
AlexDaniel [Coke]: because it is used by rakudo and by zef, and in case of zef things are really slow and can be sped up over 6 times
[Coke] ok, so once that hits, sure, private.
So instead of bundling with rakudo, bundle it with zef. 18:25
seems to avoid many of the logistic and maintenance issues.
18:26 Zoffix left
timotimo i'm hoping to be able to get a JSON::Fast version 2 on the API where we have fewer containers in the data structure 18:26
also, perhaps basing the parser on unicode codepoints rather than actually strings
lizmat timotimo: and drop support for synthetic graphemes ? 18:28
18:28 BurningPrincess0 joined
[Coke] Which makes me want to say even more, keep it in a module. 18:28
timotimo not needed
18:29 BurningPrincess0 left
AlexDaniel [Coke]: I'm not following, what do you mean by that last message? 18:31
[Coke] I would prefer we not dual home a module. 18:34
esp. one that is doing more than we need in core.
(esp if it's not going to be exposed to user-land anyway)
AlexDaniel [Coke]: so in your mind what's the solution? 18:35
[Coke] so, my recommendation is let this deprecation happen, remove the public interface; if zef needs json support, it can bundle the module as part of its own install. 18:36
AlexDaniel [Coke]: and for stuff like this we do what? github.com/rakudo/rakudo/blob/d275...m6#L24-L35 18:37
point is, both rakudo and zef currently use a json parser that is very slow 18:42
and I don't see this changing any time soon
18:42 [Tux] left
AlexDaniel “zef needs to use the same JSON parser as rakudo internals for the CUR reasons” – I don't know what these CUR reasons are, but if it's true, then there is a yet another reason to fix what we have now 18:43
it's not like we're committing to always have a copy of JSON::Fast in core, but right now it'd give an immediate, noticeable win 18:44
[Coke] should zef be doing things with internals? Or should it be using CUR: and letting it handle it?
IMO zef should not be poking at Rakudo::Internals 18:45
18:45 KombuchaKip7 joined
[Coke] AlexDaniel: parsing meta6.json doesn't seem to require any internals knowlege. 18:46
18:47 KombuchaKip7 left
[Coke] if we did want to include the module in core but not expose it, how would we do that? would it impact build time, build requirements, disk footprint, etc. 18:48
AlexDaniel [Coke]: yes? But that code is in rakudo, so what do you mean by internals knowlege?
18:49 [Tux] joined
lizmat JSON::Fast could live in lib perhaps ? 18:49
to be loaded on demand?
AlexDaniel same way we do it now
18:49 p6bannerbot sets mode: +v [Tux]
AlexDaniel I was thinking we just replace the current code with the code from JSON::Fast and call it a day :) 18:50
if timotimo agrees of course
18:55 epony left
AlexDaniel I mean, we can keep the bikeshedding on what should be done ideally go for as long as want 18:58
but it would be nice to have the speed improvement now-ish 18:59
[Coke] if you're going to hide it so it's not exposed as a module to the users, and are just replacing from-json and to-json, and the copyright is fine, sure. 19:01
timotimo i'm fine with having JSON::Fast be Artistic2, MIT, BSD3 or 2, WTFPL, LGPL, or something similar 19:02
though there's a few contributions that i didn't make
i "stole" a faster to-json (which is noted in the readme, of course)
lizmat AlexDaniel: are you still ok with accepting commits in rakudo atm ? 19:03
[Coke] (didn't write all the code, borrowed from tony-o), I would say this would need to be addressed before pulling the code into rakudo. 19:04
19:16 brrt left, epony joined, p6bannerbot sets mode: +v epony 19:25 belst3 joined 19:29 belst3 left
Geth rakudo: 12cfde2afd | (Elizabeth Mattijsen)++ | src/core/Hash.pm6
Make Hash.push/append between 20% and 40% faster

Depending on whether the key in question already existed or not		 19:31
AlexDaniel lizmat: not really :D 19:34
but ok, I'll just start a release branch
done 19:35
releasable6: vars
releasable6 AlexDaniel, branch=master; timeout=10
lizmat timotimo AlexDaniel: for zef we only need from-json, o ?
AlexDaniel releasable6: branch=release-2018.09 19:36
releasable6 AlexDaniel, branch is now set to “release-2018.09” (default value is “master”)
AlexDaniel releasable6: status
releasable6 AlexDaniel, No! It wasn't me! It was the one-armed man! Backtrace: gist.github.com/8d532938e688a2956a...6c1824fe86
AlexDaniel …
releasable6: branch=reset
releasable6 AlexDaniel, branch is now set to its default value “master”
AlexDaniel lizmat: yes, seems so. But to-json is used in rakudo 19:47
lizmat so, what is the bottleneck? from-json or to-json ? 19:49
AlexDaniel timotimo: if I'm not mistaken, that was from tony-o? And tony-o is currently willing to write a whole new parser from scratch… So I guess they won't be against including it
lizmat: from-json
lizmat I had an idea... probably stupid
AlexDaniel lizmat: at least nobody complained about to-json yet
lizmat most META6.json files are well formed 19:50
AlexDaniel lizmat: this idea? colabti.org/irclogger/irclogger_log...09-19#l293
lizmat so I was thinking of doing a very "words" based parser: if that fails, then fall back to the grammar based one
yeah 19:51
AlexDaniel JSON::Fast is not grammar based
probably not a bad idea, but JSON::Fast was already 6x as fast when I measured 19:52
tony-o [Coke]: zef relies on the internal JSON to parse things for CUR's exactly the same way the internals would 20:50
yoleaux 9 May 2018 15:56Z <Zoffix> tony-o: recall you had a tool that extracted versions from repos, based on commits to META file? Where is it? Were zef using it in the past (I thought it were)? If not, how come not? Using it would let github dists be versioned, no?
tony-o .tell zoffix sure enough. it's used on modules.zef.pm and i could probably integrate it into zef with some work, it would be slow on large repos but at least the functionality would be there i suppose 20:51
yoleaux tony-o: I'll pass your message to zoffix.
tony-o i talked to ugexe about bundling the json parser i wrote for JSON::Faster (before it merged into JSON::Fast) with zef and there were concerns about some edge cases of parsing JSON differently from internals having adverse effects 20:53
[Tux] Rakudo version 2018.08-114-g19edeafd1 - MoarVM version 2018.08-92-g3e94a68f6
csv-ip5xs0.900 - 0.906
csv-ip5xs-207.004 - 7.236
csv-parser23.714 - 24.240
csv-test-xs-200.424 - 0.430
test8.550 - 8.823
test-t2.087 - 2.116
test-t --race0.856 - 0.865
test-t-2035.796 - 37.024
test-t-20 --race11.233 - 11.341
 20:58
AlexDaniel tony-o: yeah, I'm worried about that too 21:05
tony-o ugexe brought up exposing a (de)serialize method in CU to mitigate the issue and make the internal parser private 21:11
AlexDaniel yeah, but that's a separate issue 21:12
tony-o which part is a separate issue 21:14
didn't realize we had irc logging again or i would've discussed here rather than GH issues :-) 21:15
21:19 ggoebel_ left
lizmat hmmm... my approach is 7x faster than the built-in R:I:JSON.from-jsonn 21:22
that's disappointing
tony-o 7 > 6 though :p 21:24
AlexDaniel tony-o: well, it's related, but whether we hide the internal parser behind serialize/deserialize methods or not doesn't change the fact that the internal parser should be improved 21:33
lizmat I got it up to 7.4x faster 21:38
timotimo sorry, i was AFK for big parts of the day
is JSON::Fast still on the table for replacing the internal parser?
lizmat gist.github.com/lizmat/0454f034b93...31927ca164 # my take on a pretty braindead parser 21:39
timotimo looks like it requires everything to be separated by spaces? 21:40
lizmat yeah, it assumes a well formed JSON as can be expected from a META6.json made by e.g. App::Mi6 21:41
the idea would be to put:
timotimo oh
fall back to the grammar based one
only if needed
lizmat .return with self.quick-from-json($meta) 21:42
inside the "from-json" method
timotimo there's still some things missing, but i expect you know that, too 21:43
lizmat which things? It worked on my sample META6.json
timotimo \" inside strings
i mean it has to know to abort to the safe parser in that case 21:44
lizmat do you have an example ?
AlexDaniel timotimo: yes it is 21:45
timotimo: even with quick-from-json we still don't want to fall back to turtle speeds
timotimo how much faster is quick-from-json than json-fast? 21:46
lizmat tests
21:47 BronzeEagle2 joined
lizmat timotimo: about 2x as fast as JSON::Fast, for my sample file 21:51
timotimo not terrible
lizmat hmmm... do we have a faster way to get handle \ in a string than EVAL ? 21:53
timotimo hm? 21:54
21:56 BronzeEagle2 left
lizmat something like "foo a\"bar" comes out as "'foo a\"bar' rather than 'foo a"bar' 21:56
timotimo right
really only with a trans call
well, going through character by character is also possible 22:00
but not advisable, i don't think
using index will at least be able to do 2 steps at a time 22:01
lizmat basically, removing all \ ? 22:03
timotimo well, not all \
you have to take care of double escapes and everything 22:04
lizmat hence my question: do we have some internal function that does that 22:05
timotimo not that i know
otherwise i would have stolen it for JSON::Fast :P
lizmat for a given string, rather than have to EVAL the whole thig 22:06
hehe
ok, well with EVAL it becomes *way* slower
timotimo oh yes
lizmat well, then I propose we bail on *any* \ in a string 22:08
and fall back to the slow one
timotimo that's fine
lizmat or not use this approach at all 22:09
timotimo we have to be extra sure to never attempt to parse something we can't handle with this parser 22:13
until then it's just fine to have a naive but fast parser
lizmat well, if anything, this should only be done *after* the release
AlexDaniel: can I push this to master without interfering with the release process? 22:14
AlexDaniel yes
lizmat timotimo: so, shall I put this in then?
timotimo we should try crafting some really evil json files and see if it misparses 22:15
lizmat before or after I push this ?
timotimo since it wouldn't end up in the release, it should be fine to do it after 22:16
lizmat ok, will put it in then
running spectest now 22:21
timotimo hm, so no support for numbers needed at all?
22:22 MasterDuke joined, p6bannerbot sets mode: +v MasterDuke, MasterDuke left, MasterDuke joined, herbert.freenode.net sets mode: +v MasterDuke, p6bannerbot sets mode: +v MasterDuke
timotimo interesting 22:22
lizmat when do you see numbers in a META6.json ? 22:23
timotimo right
lizmat If this works, I can probably squeeze another few percent out of it 22:24
timotimo sure 22:25
lizmat repeated item.ends-with could become a single int comparison
timotimo hm.
maybe order parse by what appears most often, which i imagine is strings
lizmat yeah, things like that 22:26
22:27 pyrimidine left
timotimo using %hash.push could get weird if a key appears multiple times 22:27
lizmat yeah, should it just be an assign? multiple keys not allowed ? 22:28
timotimo i think the json spec says nothing about multiple keys
JSON::Fast assigns
tony-o can't you just regex replace \" with " instead of EVALing lizmat ?
timotimo tony-o: we're bending over backwards to avoid regexes :D
tony-o got it 22:29
any reason for that?
timotimo gotta go fast
regex are ... not terribly fast
tony-o index('\\') and then grabbing two chars at a time doesn't seem that bad 22:30
lizmat oohh... removing the %hash.push makes it about 3x as fast as JSON::Fast
timotimo nice
because it doesn't have to check exists-key i guess
and potentially inlines better
tony-o is that gist up to date lizmat ? 22:31
lizmat no, I'll update in a mo
no, actually, will push in a mo
tony-o to blead ?
timotimo Test::Meta6 could make extra sure the json is palatable to rakudo's fast parser 22:32
lizmat tony-o: yes 22:34
japhb
.oO( COMPATIBLE-WITH-WARP-SPEED )
 22:35
MasterDuke timotimo: i probably won't be around much this evening, but if you've got a couple spare minutes gist.github.com/MasterDuke17/1c45e...d49eabd36c has the latest version of my range patch and some debug output from the only spectest fail
Geth rakudo: 16ce9c89ad | (Elizabeth Mattijsen)++ | src/core/Rakudo/Internals/JSON.pm6
Introducing the ultra-fast braindead JSON parser

This parser only handles hashes, arrays and strings and will return Nil if *anything* goes wrong. It is now plugged into "R:I:JSON.from-json" to first attempt the very fast parser: of that fails, fall back to the original, slow, grammar based parser.
How much faster: at least 10x faster!		 22:44
lizmat have fun
timotimo damn, 10x is not bad at all
lizmat is going to get some sleep
timotimo though of course that's compared to the very slow one 22:45
good work lizmat :)
and have a good rest
lizmat yeah, that last %hash.push -> %hash.BIND-KEY was the final multiplier
sleep&
AlexDaniel .tell lizmat I think you should try that parser on these files: raw.githubusercontent.com/ugexe/Pe.../cpan.json ecosystem-api.p6c.org/projects.json 22:47
yoleaux AlexDaniel: I'll pass your message to lizmat.
timotimo "authors" : "Christian S\u00e1nchez" 22:48
it'll basically immediately bail out, or at least it should
there is also coke who has his nickname in quote marks in the middle of his "author" entry 22:49
and some email addresses have the address + a little description text in quotes
er, wrong
auth being nickname \"address@foo.com\"
"name" : "Acme::\u0ca0_\u0ca0",
who did that :D
m: say "\u0ca0_\u0ca0" 22:50
camelia 5===SORRY!5=== Error while compiling <tmp>
Unrecognized backslash sequence: '\u'
at <tmp>:1
------> 3say "\7⏏5u0ca0_\u0ca0"
expecting any of:
argument list
double quotes
term
timotimo m: say "\u0ca0\_\u0ca0"
camelia 5===SORRY!5=== Error while compiling <tmp>
Unrecognized backslash sequence: '\u'
at <tmp>:1
------> 3say "\7⏏5u0ca0\_\u0ca0"
expecting any of:
argument list
double quotes
term
AlexDaniel u: \u0ca0
unicodable6 AlexDaniel, U+0CA0 KANNADA LETTER TTHA [Lo] (ಠ)
timotimo m: say "\x0ca0 _ \x0ca0"
camelia ಠ _ ಠ
timotimo ah, that's the one
liz has a few modules that have quotes in the description :) 22:51
AlexDaniel I'm also interested to know what's going to happen if you add two modules that depend on each other
afaik there's no such thing in the ecosystem yet 22:52
(because my stupid tool works)
tony-o lizmat++ 23:19
timotimo "Tadeusz \u201ctadzik\u201d So\u015bnierz", 23:20
this guy is just the absolute worst :D :D
23:39 PhonicUK joined 23:43 PhonicUK left 23:56 ggoebel_ joined 23:57 p6bannerbot sets mode: +v ggoebel_

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!