Welcome the channel on the development of Cro, a set of libraries for building reactive distributed systems, lovingly crafted to take advantage of all the Raku Programming Language has to offer (cro.services). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
08:05 sena_kun joined 10:05 Altai-man joined 10:31 sena_kun_ joined, Altai-man left 13:25 Nemokosch joined
Nemokosch Hello, I got a problem with Cro::HTTP::Client when processing a 4,5 MB JSON from a REST API. The response is resolved in around 10 seconds, however, the body never seems to resolve, not in minutes at least. 13:27
The thing is, if I request the body as string, it arrives reasonably fast, and I can parse that with JSON::Fast in again like 10 seconds
Why does await $response.body; hang, then? The header is actually application/json, by the way 13:28
The whole process finishes within 30 seconds if I do the parsing "by hand" from JSON::Fast 13:31
This is not the only weird thing. When I wrote the response to a file, it had almost 70 megabytes; if I download the same thing from the browser, as I said, it's around 4 and a half... 13:35
lizmat could it be that the request is not asking for gzip encoding ? 13:36
feels like it is about the right difference in size 13:37
Nemokosch I mean, could be, or what I could also imagine is that the browsers (Firefox at least) do limit the size of the request. 13:40
But honestly, given the nature of the problem (we want to extract data from Jira that they didn't bother to make a public API for), I can accept the bigger size as well. 13:41
The most important question is: why is that JSON::Fast can parse this huge JSON reasonably fast while await $response.body basically hangs forever? 13:42
Aha, I think some query parameter didn't get passed properly, hence my request brought back way more data. But again, I will sort this one out, this is no problem. Cro hanging on the parsing, that's a problem. 13:50
14:14 Nemokosch left 14:40 Nemokosch joined
Nemokosch Now that I did things right and only requested 5 MB, the parsing succeeded quite fast, a few seconds. Still, it would be interesting to know why the supposed JSON::Fast parsing struggled that much, when in the meantime it worked okay if I invoked it by hand. Do you think it's issue-worthy? 14:45
jnthn Nemokosch: Perhaps so, although being able to solve an issue means being able to reproduce the problem, and if it's to some URL nobody working on Cro can make a request from, that's not so easy. 14:47
It's odd in that Cro uses JSON::Fast also
So it should really be equivalent 14:48
Nemokosch Well, if it can be reproduced with any big JSON on any host, then I could just as much start a Cro server and try it :P 15:00
Meh, no, when I hosted the same file (that really makes Firefox think for some time, too), it did return... 15:09
Then I have no idea. It remains an interesting oddity. 15:10
jnthn There is a CRO_TRACE=1 but I fear it'll dump out so much data on such a big file it won't tell us so much 15:25
(as an env var, I mean)
16:11 Nemokosch left 16:13 sena_kun_ left 16:14 sena_kun_ joined 17:06 sena_kun_ left 19:45 sena_kun left 19:47 sena_kun joined 21:02 sena_kun left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!