Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:43 MasterDuke joined
MasterDuke . 00:55
tellable6 2024-04-17T08:25:57Z #moarvm <lizmat> MasterDuke: sorry for the noise
MasterDuke jnthn, nine, etc: while reducing the string heap might not produce much of a measurable benefit, what about the fact that converting cuuid from an MVMString to and MVMuint64 removes a GCed field from MVMStaticFrame? think that would have a noticeable enough benefit to keep on pursuing this? 00:59
and if so, any suggestions on how to bootstrap/propagate-up-through-nqp-and-rakudo  the change? 01:24
01:28 bloatable6_ joined, unicodable6_ joined, nativecallable6_ joined, evalable6_ joined, squashable6 joined, coverable6_ joined, releasable6_ joined 01:29 bloatable6_ left, unicodable6_ left, releasable6_ left, coverable6_ left, evalable6_ left, squashable6 left, nativecallable6_ left 01:30 tellable6_ joined, committable6_ joined, shareable6_ joined, sourceable6_ joined, statisfiable6 joined 01:31 linkable6_ joined, nativecallable6_ joined, quotable6_ joined, benchable6_ joined, unicodable6_ joined, notable6_ joined 01:32 evalable6_ joined, greppable6_ joined, squashable6 joined, bloatable6_ joined, releasable6_ joined, coverable6_ joined, bisectable6_ joined 01:33 reportable6 joined 01:40 evalable6_ left 01:41 evalable6_ joined 03:58 MasterDuke left
Woodi yesterday I was watching FOSDEM2024 presentation about dashboard using duckdb and for performance reasons they changed strings into ints. they have data, on load to db they convert to enums and use that in queries 04:25
maybe it is inline::perl which needs to be converted ? :) 04:26
also: it is start up so everything matters... 04:58
05:00 lizmat__ joined 05:03 lizmat left 05:53 lizmat joined 05:57 lizmat__ left 06:00 reportable6 left 06:02 reportable6 joined
nine MasterDuke: still doubt that would bring any measurable benefit. Mind that I could also just be wrong. 06:52
tellable6 nine, I'll pass your message to MasterDuke
06:53 lizmat_ joined
nine Woodi: I'm quite sure they had several orders of magnitudes more strings in that database than we're talking about here :) 06:53
06:56 lizmat left
Woodi sure, but MVMuint64 cuid_idx = get_vm_string_index(tc, ss, MVM_coerce_i_s(tc, sf->body.cuuid)); or just X x 20k array walking on startup sound bad. and how many pointers dereference and function calls ? just saying: nice to consider... 08:07
btw. that are strings related to source code and transfered into arryas ? 08:10
nine Again, there is certainly a cost here. I'm just saying that 20K things may sound much to a Raku or Perl programmer, but for C code that's practically nothing. Would it be worth the effort if all it does is save some 0.2ms on startup? 08:18
Woodi nine: hom much is 0.2 ms :) as Counter-Strike player if it is noticable to eyes then it is worth :) also as cli user: if makes difference for: command ENTER ARROWUP ENTER then worth. but that just my use case :) 08:25
nine Of course I can't tell anyone what to do or not to do. I wouldn't even want to. Just saying that if it were my time, I'd want to see a profile first that gives me some indication that this optimization might be worth it.
Woodi sure. 'MVMuint64 cuid_idx = get_vm_string_index(tc, ss, MVM_coerce_i_s(tc, sf->body.cuuid));' is that code what we talking about or lines ? 08:27
s/other lines/
nine 0.2ms is incidentally about 0.2 % of rakudo -e '' on my machine. So no, you wouldn't notice it at all.
That get_vm_string_index call is from the profiler.
08:30 lizmat_ left
Woodi ack says get_vm_string_index is used 4 times so 0.8ms ? :) 08:31
08:31 lizmat joined
Woodi wait, profiler ? it is compiled in by default ? 08:32
that arrays are probably actual optimization :) 08:34
nine get_vm_string_index is a static function in src/profiler/heapsnapshot.c. It therefore can only be used within that file and it is only used when the profiler is active 08:35
Woodi and when profiler is used ? it is part of JIT or something or profiled, one time compilation ? 08:37
nine MasterDuke: come to think of it, chances are good that thans to lazy deserialization we won't ever load most of these static frames and their strings. It'll be a rare program that uses everything that is available in the setting. And the things we do load get put into gen2 directly, so they won't be touched by most GC runs. 08:38
tellable6 nine, I'll pass your message to MasterDuke
Woodi get_vm_string_index is called twice in set_static_frame_index which is part of finalization... 08:39
nine The profiler is only used when you actually run your program with rakudo --profile. And this particular part only if you run with --profile-kind=heap 08:40
Woodi ah, so not very one-line use case...
08:48 sena_kun joined
Woodi do that warnings are misleading ? src/disp/inline_cache.c:629:19: warning: array subscript ‘MVMDispInlineCacheEntryPolymorphicDispatchFlattening[0]’ is partly outside array bounds of ‘MVMDispInlineCacheEntry[1]’ [-Warray-bounds] 09:48
nine Yes, looks bogus 09:51
Woodi hmm, nqp have option --nqp-home so I can have --with-moar=/pkg/moar202403 and /pkg/nqp202403 ? possibly /pkg/rakudo202403 too ? and then files in /pkg/rakudo202403/bin/ will work ? 10:08
nine There's an NQP_HOME environment variable 10:11
Woodi MVMDispInlineCacheEntryPolymorphicDispatchFlattening isn't that overengineered ? :) what language need this ?? ;)
so add it to ~/.profile ?
so what is difference between --prefix and --[nqp|rakudo]_home options ? Configure.pl says "dir to install files"... 10:43
lizmat patrickb might know 11:38
Woodi looks, in 3rdparty/nqp-configure/lib/NQP/Config.pm that *_home are opposite to relocatable whatever relocatable means for non os-executables. like "portable executables" maybe - just unpack anywhere and use, maybe ? with env vars possibly... 11:49
12:00 reportable6 left 12:01 reportable6 joined
patrickb prefix is where the installation root dir is during make install (e.g. /usr/local). nqp/rakudo home are the folders where the rakudo/nqp installations are to be found at runtime. (e.g. /usr/local/share/nqp) I don't remember, are there rakudo/nqp_home options of configure.pl? 12:37
With relocatable installations the raku/nqp executables locate the rakudo and nqp home dirs at runtime, so they are not hard coded. 12:39
I think with the --rakudo_home var one can override the default path (based on the prefix). 12:40
It's possible to override the dir at runtime using the env vars. 12:41
12:46 lizmat_ joined 12:49 lizmat left 12:50 lizmat_ left, lizmat joined
Woodi question is when env vars override, always (configure names *_home static) or with relocable build ? also do *_home override everything from --prefix ? 13:28
I will try few configurations, maybe separate builds can be packaged nicely 13:29
13:59 lizmat_ joined 14:03 lizmat left 17:32 lizmat_ left, lizmat joined 18:00 reportable6 left 18:02 reportable6 joined
lizmat still working on refining MoarVM::Bytecode, and found another interesting pattern 18:23
many annotations are identical 18:24
e.g. a stub method such as: method pull-one() { ... }
generates three identical annotations
all pointing at the same bytecode offset and line number 18:25
(all annotations of a frame share the same filename anyways, as expected :-)
nine Well that string will at least have been interned, so we're only storing it once and just referencing it multiple times. 18:45
lizmat looks like the c setting has 28210 identical annotations, at 12 bytes each -> 338K of memory 18:48
it's all memory mapped, and I'm pretty sure the annotations are really only used in special occasions, like backtraces and such 19:09
related: since a frame can only exist in one file (afaik), it's seems a bit wasteful to keep a string index for each statement, pointing to the same file string 19:11
rather than having a single string index in the frame information itself
just like the name of the frame 19:12
(which I doubt is needed during execution either?)
jnthn It was done like that for static inlining and macros, fwiw 19:56
lizmat so I guess the name of the frame *is* needed for execution, as its index is part of the frame structure ? 19:57
jnthn But we don't do much at all of the former (only for natively typed things)
There's an op for getting the name and I think it is used in some places
Can't remember exactly where 19:58
lizmat nqp::getcodename ?
jnthn Sounds like
lizmat hmmm in the Rakudo src I can only find it used for introspection 19:59
unless FAIL-GOAL is execution? 20:00
jnthn Well, it's called at runtime at least 20:12
I think it was used in more places, e.g. action method calls
But frame introspection frustrated inlining tehdy 20:13
lizmat tehdy ?
terribly?
jnthn oops! at that time
I think nowadays nqp::curcode and the like actually are inlinable 20:14
But various uses of nqp::getcodename that used to be there probably went away at some point due to that limitation 20:15
22:02 lizmat_ joined 22:06 lizmat left 22:56 lizmat joined 23:00 lizmat_ left 23:31 lizmat_ joined 23:35 lizmat left, lizmat__ joined 23:38 lizmat_ left, lizmat joined 23:42 lizmat_ joined, lizmat__ left 23:45 lizmat__ joined 23:46 lizmat left 23:48 lizmat joined 23:49 lizmat_ left 23:51 lizmat_ joined 23:52 lizmat__ left 23:55 sena_kun left, lizmat left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!