|
00:52
librasteve_ left
13:23
disbot1 left,
disbot1 joined
13:28
ds7832 joined
|
|||
| antononcube | @theoriginalwaffle I assume there is a similar Perl package ("Net::AMQP"). If yes, does that Perl package adhere to your expectations? Is there a "max-message-count" argument that might be set to, say, 80_000? | 14:22 | |
|
15:53
arkiuat joined
|
|||
| theoriginalwaffle | @antononcube There is a Perl package Net::AMQP::RabbitMQ which says is a wrapper around librabbitmq. I've not tried it but expect that since it is a wrapper around a lib that it would behave differently. I don't see any max-message-count type of argument and something like that set as a default would be weird. I also tried this example earlier today: | 15:56 | |
| github.com/jonathanstowe/Raku-RMQ-...torial-002 and it doesn't work. The send component does send a message, but the receive just pulls the message and hangs. It should print and ack the message, but does not. | |||
| antononcube | Ok. What is frame-max here? github.com/jonathanstowe/P6-Net-AM...QP.pm6#L16 | 16:15 | |
| theoriginalwaffle | Seems to be a frame size used to keep a publisher of very large messages from impacting other connections. RabbitMQ docs say this "Maximum AMQP 1.0, AMQP 0-9-1 and RabbitMQ Stream Protocol frame size. Should not be changed; rely on server and client library defaults." | 16:40 | |
| Looks like it is being set during the connection handshake with the server. Like a connection.tune method is being sent from the server and the frame-max is part of that. github.com/jonathanstowe/P6-Net-AM...QP.pm6#L65 | 16:48 | ||
| antononcube | @theoriginalwaffle From what you say, it seems it is better to make a "Net::AMQP" version -- e.g. Net::AMQP::RabbitMQ" -- that uses the library you mentioned ("librabbitmq"). | 17:20 | |
|
17:26
arkiuat left
|
|||
| theoriginalwaffle | @antononcube Yeah, I was thinking about something like that. | 17:29 | |
|
17:39
arkiuat joined
17:45
arkiuat left
17:47
arkiuat joined
|
|||
| Hmm.. I wonder if it is this start block: github.com/retupmoca/P6-Net-AMQP/b...e.pm6#L253 that is causing the cpu/ram issue. I got the Stowe example working, but now it is slow like I posted above.. could the start block be building up a huge backlog of async processes waiting to be executed? | 17:49 | ||
| I'm going to remove that and see what happens.. 🙂 | 17:50 | ||
| lizmat | what are you trying to achieve with the emit in a start block ? | 18:09 | |
| I mean, it's already in a tap ? | 18:10 | ||
| if it is because you want to return as quick as possible, then why not set up %headers inside the start block as well? | 18:11 | ||
|
18:22
librasteve_ joined
|
|||
| theoriginalwaffle | I'm not sure the intention but it looks to me like it could cause the problems I am having so I am going to remove it and see how it goes. | 18:25 | |
|
18:26
ds7832 left,
Guest3246 joined
|
|||
| Nope, removing the start block didn't make a significant (if any) difference. Starts consuming kinda fast, and then just slows down, ram grows, cpu at 100%. | 18:40 | ||
| lizmat | this is generally an indicating of not enough backpressure | ||
| *indication | |||
| medium.com/@jayphelps/backpressure...50b3e77ce7 | 18:43 | ||
| theoriginalwaffle | Yeah it kinda smells like that, but in this case there are messages on the RabbitMQ server, about 10k of them for testing, and then the producer is disconnected. So the messages are just there in the queue waiting to be processed. The consumer connects and stars pulling messages, and that is what I am having issues with (also some issues on the producer, but keeping that out-of-scope for now). | 18:44 | |
| lizmat | the handling of incoming messages is drowning out the handling of them, it appears | 18:45 | |
| s:1st/handling/accepting/ | |||
| theoriginalwaffle | The consumer is doing nothing but printing the message body (the number of the message so one number from 1 to 10k) and acking the message. So that it starts up kinda fast, but quickly slows down (to about 3 msgs>/sec after a min or so of running) uses 100% processing those 3 msgs/sec and ram keeps growing. | 18:47 | |
| lizmat | removing the start didn't help, you said | 18:48 | |
| theoriginalwaffle | It is reminding me of times I have seen have some data structure, like an array or something, that is growing without bounds. Not getting cleaned up and then accessing elements of it gets slower and slower. | ||
| Right, removing the start didn't help. | |||
| I thought that might be it... if the async processes were building up without bounds. | 18:49 | ||
| lizmat | why is the $delivery-lock.protect in there ? | ||
| theoriginalwaffle | No idea. This module is the only RabbitMQ module for Raku and I just started playing with it yesterday and noticed the issues. | ||
| lizmat | perhaps check some of the git blame of the lines in question? | 18:53 | |
| sadly no inline docs in that module :-( | 18:54 | ||
| sorry, don't think I can be of more help atm | 18:58 | ||
| theoriginalwaffle | Nothing standing out to me in the git blame. I'm also not understanding the $delivery-lock.. Is it trying to make sure the headers are processed first? | 19:01 | |
| No worries. Thanks! | 19:02 | ||
|
20:23
Guest3246 is now known as ds7832,
ds7832 left,
ds7832 joined
20:32
librasteve_ left
20:35
arkiuat left
20:39
arkiuat joined
20:44
arkiuat left
20:47
arkiuat joined
20:57
arkiuat left
21:10
arkiuat joined
21:15
arkiuat left
21:43
arkiuat joined
21:48
arkiuat left
21:51
arkiuat joined
21:56
arkiuat left
|
|||
| I was looking into the $delivery-lock some hours ago and thought I'd write up my last thoughts before heading out. It seems like it is fine. I am pretty certain the reason is just to make sure the headers are processed first. A lock is acquired within the scope of both $htap and $btap but then only the $htap unlockes it. That keeps the $btap from rumming. Not going to make a judgement call about whether it | 22:08 | ||
| makes sense or not, but that seems to be the intention. | |||
|
22:17
arkiuat joined
22:26
arkiuat left
22:30
arkiuat joined
23:53
ds7832 left
|
|||