|
github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm Set by AlexDaniel on 12 June 2018. |
|||
|
00:02
reportable6 left,
reportable6 joined
00:40
MasterDuke left
04:15
japhb left
04:17
japhb joined
06:02
reportable6 left
06:03
reportable6 joined
06:47
squashable6 left
06:48
squashable6 joined
07:51
domidumont joined
08:06
zakharyas joined
08:07
frost-lab joined
09:33
Kaeipi joined
09:34
Merfont left,
Kaeipi left
09:35
Kaeipi joined
09:41
Kaeipi left,
Kaiepi joined
10:53
zakharyas left
12:01
reportable6 left
12:03
reportable6 joined
13:07
zakharyas joined
|
|||
| lizmat | So I think I found a reason why some applications eat much more memory than expected | 13:21 | |
| on logs.liz.nl all of the raku logs are read, but the original raw log was *not* kept around | |||
| a lot of substr into the raw log are | |||
| to shortcut searching, I figured it could be beneficial keeping the original logs in memory | 13:22 | ||
| to my surprise, this did *not* affect memory usage in any notable way | |||
| so my conclusion is, is that a substr into a string *is* keeping the original string alive | 13:23 | ||
| is that a correct assumption ? | 13:28 | ||
| jnthn | Yes | 13:37 | |
| lizmat | and there is no way to "extract" a string from its "source" ? | 13:58 | |
|
14:14
frost-lab left
|
|||
| jnthn | encode/decode | 14:18 | |
| Will certainly do it, of course at a cost | |||
| .NFC.Str too, for a lower cost | |||
| lizmat | aha,, ok, fair enough...something to keep in mind | 14:19 | |
| I was originally thinking about instead of keeping substrings, just keeping the raw log and offsets/lengths into that | 14:20 | ||
| now this is clear to me, I realize the system is already doing that for me :-) | |||
| timotimo | upon garbage collection, we may want to sometimes check if a very long string is being kept around just for a comparatively short substring | 14:49 | |
| jnthn | MoarVM already does expose a way, iirc, to "flatten" a strand string into a single thing, and we could potentially add a named argument to substr like :copy to force it to avoid the optimization in question | 14:51 | |
| And implement it using that | |||
| Or pass it down somehow | |||
| timotimo | yeah nqp::indexingoptimized is the op | 14:55 | |
| ideally nobody would have to think about this :D | |||
| lizmat | well, yeah... actually, now I'm actually keeping the raw log accessible, I don't need it anymore :-) | 14:59 | |
| timotimo | i wonder if there should be access to the rope datastructure's details for users at all | 15:10 | |
| that sounds like there is now but i'm saying there shouldn't be | |||
| i mean, i wonder if it'd be good if users could access the rope's pieces | |||
| ugexe | is that even possible for non-moarvm backends? | 15:12 | |
| timotimo | no, they would just say "this has one piece, it has 0 repetitions, it starts at 0 and ends at n, and the target string is $self (maybe) | 15:13 | |
| " | |||
| i guess that could be trouble if you iterate over it? | |||
| there'd have to be a signal for "this is a terminal string"? "this is a leaf"? | 15:14 | ||
| lizmat | please note I'm fine with how it is now | ||
| and it *is* an implementation detail | |||
| timotimo | mhm | ||
| lizmat | it's just that it is an extra reason when you're extracting information from a log file, using substr and all | 15:15 | |
| it's probably better *not* to slurp the whole file first | |||
| timotimo | even if you slurp it line-by-line, you'd keep each line around if you just hang on to a small substring | 15:16 | |
| lizmat | yeah... there's that | ||
| hmmm | |||
| so, if I would run nqp::indexingoptimized on the slurped file contents | 15:17 | ||
| and then discard it | |||
| timotimo | you'd be expected to call it on everything you substr'd out of it | ||
| lizmat | ah, ok, so that doesn't make sense then :-) | ||
| timotimo | it's a little bit of work ;) | 15:18 | |
| overwrite the substr sub/method to call indexingoptimized on the result, that's how :copy would be implemented | |||
| lizmat | yeah, got that | 15:33 | |
| I'm not sure we should, though? | |||
| previous discussion: github.com/rakudo/rakudo/pull/3975 | 15:35 | ||
|
16:16
moon-child left
16:21
moon-child joined
17:15
domidumont left
17:53
zakharyas left
18:00
cog_ left
18:02
reportable6 left,
reportable6 joined
18:53
cog joined
18:59
lucasb joined
19:09
squashable6 left
19:10
squashable6 joined
19:11
cog left
19:12
cog joined
19:43
MasterDuke joined
20:09
rypervenche left,
jpf1 left,
rypervenche joined,
jpf1 joined
20:38
MasterDuke left
20:44
MasterDuke joined
21:38
MasterDuke left
|
|||