| lizmat | timo: #masakism logs now also online | 00:20 | |
| sleep& | |||
| jubilatious1_98524 | Hi @shimmerfairy , I nudged an ICU conversation on the old perl6-users mailing list, and got 13 replies (latest 2024). Here take a look: www.nntp.perl.org/group/perl.perl6...g9241.html | 04:55 | |
| lizmat | m: .say with try 56792.chr # sorta expected to have that say Nil | 11:41 | |
| camelia | Error encoding UTF-8 string: could not encode Unicode Surrogate codepoint 56792 (0xDDD8) in block <unit> at <tmp> line 1 |
||
| lizmat | actually, not say anything at all :-) | ||
| timo: for reading large (2GB+) files, would it make sense to presize the Buf, (and then resize to what's used) and then have nqp::readfh start adding bytes *after* what is already in there? | 12:57 | ||
| timo | it makes sense that the 56792.chr works but the say doesn't; after all, our strings are capable of more than just™ utf8, so it can represent a lone surrogate codepoint ... or maybe we don't want that to be possible at all is what you're suggesting? | 13:33 | |
| patrickb | I'm puzzled again. Looking at github.com/MoarVM/MoarVM/blob/main...ode.c#L764 The frame deserializer only reads the debug_locals data if the debug server is active, but it does not advance `pos` when the debug server isn't active. | 13:36 | |
| Won't this mess up all following deserialization? (Or is it fine simply because it's the last part of the frame data?) | 13:37 | ||
| timo | it's the last part, the next frame's position will be read from the table of all frame's positions again | 13:38 | |
| patrickb | Ah. Understood. Then it's fine. Thanks for confirming! | 13:39 | |
| timo | no problem! | ||
| actually, a few months ago I hacked together an imhex pattern definition file for moarvm files, which you may find interesting to look at. also there's lizmat's module that can read these files and give statistics and such | 13:40 | ||
| lizmat | MoarVM::Profile you mean ? | ||
| timo | no i think it's MoarVM::Bytecode? | 13:42 | |
| lizmat | ah, ok, yes | ||
| Q: given enough RAM, should we be able to slurp a 2G+ file ? | 13:48 | ||
| timo | slurp to string, right? | 13:53 | |
| lizmat | nope... binary | 13:54 | |
| sorry | |||
| timo | yeah that should surely work. do we still grow buffers only linearly after a specific size? | ||
| lizmat | well, I'm working on optimizing that | 13:55 | |
| timo | ISTR a pull request or maybe just a branch with experiments for that | ||
| lizmat | yeah, that got merged, but it also needs some rework | 13:56 | |
| timo | what would be really nice is if we could mmap a file into a Buf, but that's actually a different operation from slurping, since then changes to the file will cause changes in memory as well | ||
| lizmat | well, I'll leave that for a later exercise | ||
| timo | you can already get that with NativeCall and a CArray or CPointer | 13:58 | |
| lizmat | % rl '"m5.sql".IO.slurp' | ||
| initial read of 1048576: 1048576 | |||
| taking slow path for 2417660750 | |||
| sub slurp-PIO-uncertain(Mu \PIO, int $size) | |||
| trying to read 1879048192 bytes | |||
| appending 538612558 to 1879048192 | |||
| trying to read 1879048192 bytes | |||
| MoarVM panic: Memory allocation failed; could not allocate 18446744066204519736 bytes | |||
| timo | well, that's not great! :) | 13:59 | |
| lizmat | feels like that allocation number is... way too big ? | 14:00 | |
| timo | yes, negative number | ||
| I recall looking at some code paths where we made sure to use the biggest number type for file size information the OS / C library can give us | |||
| m: say 18446744066204519736.base(16) | |||
| camelia | FFFFFFFE40AA4D38 | ||
| lizmat | m: my int $ = 18446744066204519736 | 14:01 | |
| camelia | Cannot unbox 64 bit wide bigint into native integer. Did you mix int and Int or literals? in block <unit> at <tmp> line 1 |
||
| lizmat | anyways, will contemplate while doing some gardening& | 14:03 | |
| timo | maybe check with strace if you can spot where the number may come from | 14:05 | |
| patrickb | When a raku process (my debugger UI) stops and doesn't perform any cleanup on shutdown, a Proc::Async child (the moar under debug) might first receive a SIGHUP (causing moar to try to shutdown) and then a SIGINT (causing it to die immediately)? I'm asking because when shutting the UI down I receive a last sole 83 byte from moar. I'd guess it's the start of a ThreadEnded notification. Does that sound plausible? | 15:39 | |
| Byte 83 is the introducer of a fixed with map with 3 elements. | 15:40 | ||
| timo | I can only recommend rr to record the whole process tree so you can figure things like that out reliably :) | 15:58 | |
| sigint shouldn't kill moar immediately, but we also don't catch it by ourselves so we really don't do very much between receiving sigint and exiting | |||
| lizmat | an update on the MoarVM panic: Memory allocation failed; could not allocate 18446744066204519736 bytes error | 16:16 | |
| if I *don't* do the $blob.append($part) I don't get an error | 16:17 | ||
| so it feels like the nqp::splice of large buffers is doing something wonky | |||
| aha: looks like it fails when such a large blob is being returned from a sub | 16:46 | ||
|
16:48
[Coke]_ joined
|
|||
| lizmat | frobnicating& | 16:48 | |
|
16:51
[Coke] left
|
|||
| timo | if we are adding newly read blobs to an existing blob one by one, if we don't otherwise have GC pressure (did not verify) we might keep a lot of smaller blobs around before gc can toss them out? | 17:34 | |