Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:03 quotable6 joined, greppable6 joined, committable6 joined 00:04 shareable6 joined 00:05 linkable6 joined, nativecallable6 joined, tellable6 joined, coverable6 joined 00:16 tealecloud left 01:43 moon-child left 01:44 moon-child joined 02:03 unicodable6 joined, statisfiable6 joined 02:04 squashable6 joined 02:05 reportable6 joined
japhb How solid is MoarVM cross-compile support? I see some bits in Configure.pl for it, but I don't know how recently that's been run through its paces ... 02:06
02:13 squashable6 left 02:14 squashable6 joined
japhb Also, is anyone (lizmat perhaps?) building on MacOS on Apple M1? 02:16
03:14 evalable6 left, linkable6 left 03:18 psydroid left 03:23 psydroid joined 03:27 ggoebel left 03:28 squashable6 left 04:16 evalable6 joined 04:30 squashable6 joined
timo hm, how do you CI cross-compilation, i wonder. i would say qemu but that's slooooow 05:09
05:17 linkable6 joined 06:02 reportable6 left 06:04 reportable6 joined
japhb Requesting early feedback, to take advantage of timezone differences: gist.github.com/japhb/0c2108affd31...b3eb6a7947 06:47
PLEASE NOTE: This is not anywhere *close* to ready, and is missing a lot of detail, examples, resource links, etc. I already know this. :-)
I'm just trying to determine if there are major porting risks or task categories that I'm missing. 06:48
Nicholas good *, japhb 07:18
I've got shell acces to an M1 Mac - everything builds quite boringly there.
japhb \o/ 07:19
That's very good news.
Nicholas: Do you happen to know if the C ABI is different under MacOS/AArch64 than Linux/AAarch64? 07:20
moon-child I believe it is 07:21
at least: varargs are all passed on the stack, and you are not allowed to use x18
there may be more differences
japhb Dang, that's annoying. OK, thanks for the info, moon-child.
moon-child 'Because x64 (and x86 before it) are among the most alignment-lax architectures in common use' no need to sound so derogatory--unaligned accesses have proven themselves well worth the silicon :) 07:23
(that said I think modern arm is also fairly tolerant of unaligned accesses in most modes) 07:24
japhb moon-child: Oh hmmm, I hadn't meant that as derogatory, more just a simple statement of fact. 07:25
Nicholas Is "alignment tolerant" better? 07:26
moon-child was mostly just poking. Hence the :) :)
japhb How about "Because x64 (and x86 before it) have been the primary 07:27
MoarVM development platform so far and are very alignment-tolerant,"
moon-child: It was a totally valid concern. :-) 07:28
Nicholas you use "forgiving" later to describe the x86 memory model. I think that that's probably a better word than "lax" or "tolerant" 07:29
japhb Nicholas: šŸ‘ 07:33
07:33 patrickb joined
patrickb japhb: I also own an M1 Mac. It's currently only used for building the compiled rakudo releases. I intend to set it up to allow remote ssh access, but simply didn't manage to do so yet. 07:34
japhb patrickb: Ah, good to know, thank you
Oh, forgot to respond to one of your comments, moon-child -- It is possible that the most recent AArch64 variants have relaxed some of the alignment strictness (wouldn't surprise me, actually -- as you point out, it can be a good use of silicon), but I'm taking the view that supporting all AArch64 variants back to the original ARMv8-A introduction (but nothing that isn't a superset thereof) is a good place 07:38
to draw a line that captures most of the ARM systems we would reasonably want to JIT today.
moon-child it appears armv8-a supports unaligned accesses too. At least, stackoverflow.com/questions/457145...on-aarch64 says it does 07:41
(that said taking a longer view towards wider architecture support, it might be good to figure out how to support stricter alignment requirements. Gcc famously generates bad code when alignment requirements are high) 07:42
japhb (If anyone thinks ARMv8-A is the wrong line, please do say so -- but I figure it's way easier for us to say "We support all AArch64 variants" than confuse users with "Well, if you're running ARMv8.5-A, ARMv8.6-A, or ARMv9-A then yes, but ...") 07:43
Interesting regarding the unaligned access. Some of the instruction encodings seem to directly add trailing 0 bits based on the operand size (so it's literally not possible with those encodings to refer to unaligned data), but perhaps experienced ARM coders know how to avoid those instruction encodings efficiently. I'll have to look more into that. 07:46
patrickb japhb: I'd like to know how much work it would be to get rid of the lego JIT. I think the intention has long been to get rid of it. If it's not that far off, it might make sense to tackle that first. 07:47
japhb patrickb: Right now the expression jit depends on some of the infrastructural parts of the lego JIT (as well as being a fallback in case expression JIT cannot handle some ops). That infrastructure part probably could be disentangled first. 07:51
.tell brrt Saw your comment on the gist, thank you, will reply in the morning as my brain is currently fading a bit.
tellable6 japhb, I'll pass your message to brrt
Nicholas the only thing that I have access to that builds MoarVM that is really grumpy about alignment is sparc64 07:57
ppc64 can cope with 64 bit loads (at least at) 32 bit alignment
so if there are 64 bit ARMs that aren't happy about 64 bit integer loads (more) off-alignment than we currently do, I've not hit them 07:58
I'm not familiar with the 64 bit ARM instruction set, or what alignment you start having
moon-child why do you have a sparc?
Nicholas but even x864_64 gets grumpy (SIGsomething) if you attempt to go "off alignment" with SIMD instructions. And if you compiler vectorised a loop, working sloppy code can go boom 07:59
japhb 16-byte alignment (what x86 used to call "paragraph alignment") for the AArch64 stack was the biggest surprise I've come across so far.
Nicholas I don't. I have access to one. It's cfarm.tetaneutral.net/machines/list/
japhb OK, it's past 1 AM here; I'm going to wander off to the land of nod for now, but will backlog any further comments/suggestions when I'm back online. 08:02
Goodnight, y'all! 08:03
&
08:10 tealecloud joined 08:47 Xliff left 08:51 patrickb left 09:08 ggoebel joined 09:12 patrickb joined 09:18 patrickb left, patrickb joined 09:51 ggoebel left
jnthnwrthngtn moarning o/ 09:54
Nicholas \o
10:07 linkable6 left 10:09 linkable6 joined 10:14 patrickz joined 10:15 patrickb left 10:22 lizmat_ joined, Geth_ joined, RakuIRCLogger joined 10:23 TempIRCLogger joined 10:24 RakuIRCLogger_ left, Geth joined, TempIRCLogger__ left, lizmat left, Geth left, Geth_ left 10:25 Geth joined, lizmat_ left, lizmat joined
jnthnwrthngtn Goodness, the NQP build is master than the MoarVM build with Visual C++ / nmake... 10:31
uh, faster 10:32
lizmat
.oO( time to drop C as an implementation language :-)
 10:34
jnthnwrthngtn :D 10:38
Nicholas you can fix this with ASAN 10:39
jnthnwrthngtn Anyways, the build succeeded, which is good but also bad because I failed to repro what [Coke]++ is observing and have few guesses left of what it could be 10:40
dogbert17 jnthnwrthngtn: didn't [Coke]++ mention that some nativecall tests failed on windows? 10:43
jnthnwrthngtn Only with a debug build 10:45
dogbert17 ah
10:46 sena_kun joined
jnthnwrthngtn Hm, wonder if the debug server problem I'm looking at is a regression everywhere or just a Windows-specific issue... 10:46
sena_kun jnthnwrthngtn, you mean it not connecting to the socket or something new? 10:47
jnthnwrthngtn It seems to connect but doesn't hit a breakpoint; on Linux it does hit it, but then hangs when I resume. 10:49
sena_kun ouch
good luck
jnthnwrthngtn Pretty sure that's a regression
It's the simplest possible example
ah, it's backslash vs forward slash confusion on Windows 11:10
Geth MoarVM/debug-server-fixes: ab2a3f01c2 | (Jonathan Worthington)++ | src/debug/debugserver.c
Normalize filenames for debug server

So that we can avoid missing breakpoints due to / vs. \ on Windows.		 11:22
MoarVM/debug-server-fixes: 137ca8343e | (Jonathan Worthington)++ | src/instrument/line_coverage.c
Produce one breakpoint instruction per line

Some lines span multiple basic blocks, which would result in us placing multiple breakpoint instructions. This could lead to the debugger stopping many times on the same line.		 11:36
12:02 reportable6 left 12:04 reportable6 joined 12:12 cognominal joined 13:04 tealecloud left
Geth MoarVM/debug-server-fixes: b5b276ffd1 | (Jonathan Worthington)++ | 2 files
Fix issues with resuming suspended threads

  * The case of being unable to GC and with an outstanding suspend
   request was not handled
  * A mutex was not acquired while using a condition variable, leading to
   lost wakeups
  * For efficiency, we also only broadcast the condition variable after
   going through all of the threads to resume them		 13:22
nine Looksblight a mighty yak shaving trip 13:37
Looks like
[Coke] jnthnwrthngtn: thanks for trying to repro my windows issue;. 13:43
patrickz jnthnwrthngtn: Do these debugger fixes fix bugs new-disp brought or were they potentially present in 2021.09 as well? 13:46
Geth MoarVM/debug-server-fixes: 8f7d970aed | (Jonathan Worthington)++ | src/gc/orchestrate.c
Only produce debugger debugging output when asked
 13:47
jnthnwrthngtn patrickz: These all seem to go back quite a long way
nine: The yak is "argh, we have too many bug reports about the Comma debugger", pretty much. 13:48
nine Which means that it gets used :) 13:49
patrickz OK. I haven't had much luck using the debugger until now. Seeing those fixes makes me happy. jnthnwrthngtn++ 13:50
13:52 brrt joined
Nicholas good *, brrt 13:52
patrickz o/
Nicholas brrt: I was wrong. "JIT all the things" is so last week. it's now "dispatch all the things" 13:53
brrt good * Nicholas
tellable6 2021-10-06T07:51:35Z #moarvm <japhb> brrt Saw your comment on the gist, thank you, will reply in the morning as my brain is currently fading a bit.
brrt ah
that's good
MasterDuke brrt: have you seen my recent jit-related PRs and the problems i've been having on windows? 13:54
brrt I have not 13:56
which are the ones?
though I'm vaguely aware
MasterDuke github.com/MoarVM/MoarVM/pull/1555 currently is ok 13:58
but github.com/MoarVM/MoarVM/pull/1552 has some extra ops that fail on windows 13:59
scsetobj, scsetcode, getcomp, and setdebugtypename
Geth MoarVM/debug-server-fixes: 746175bc9a | (Jonathan Worthington)++ | src/debug/debugserver.c
Fix regression in stepping
 14:28
MoarVM/debug-server-fixes: cedebdbf7f | (Jonathan Worthington)++ | src/debug/debugserver.c
A little more debug output for stepping
MoarVM: jnthn++ created pull request #1559:
Assorted debug server fixes		 14:33
brrt I'll have a look 14:41
[Coke] so I rebooted this morning... and I think things are now failing *differently* 15:06
doing a re-rebuild with VERBOSE_BUILD to verify
15:33 brrt left 15:34 brrt joined
Nicholas [Coke]: if you pour coffee into the computer do your problems go away? Or do you just get a whole new set of different problems? :-) 15:36
jnthnwrthngtn I've only tried this experiment with a keyboard, but it didn't make much difference. 15:40
nine Oh the coffee is supposed to go into the computer?! I've always... well better not say where I put it 15:41
timo jnthnwrthngtn: i'm having trouble coming up with a sensible solution to the argument spesh being aborted when a NativeRef is passed and it desires something that boxes an int, unless we teach spesh that Int*Ref can in fact box int 15:45
[Coke] gets past where it was. now dying on instal with: 15:46
C:\sandbox\rakudo\rakudo-m.exe" --ll-exception "C:\sandbox\rakudo\tools\build\install-core-dist.raku" "C:\raku\share\perl6\core" 15:47
... which seems super familar. :|
gist.github.com/coke/2aa9eaa797004...5a10c18893
(this is with rakudo HEAD and recommended nqp/moarvm) 15:48
jnthnwrthngtn `nmake install` did succeed for me (otherwise I'd not have had a working install to debug the debugger) 15:49
And I was doing it with a 64-bit environment so there's JIT 15:50
[Coke] it bugs me that a reboot fixed the last issue. :) 15:51
nine Ah, the error: no error again? 15:52
[Coke]: have you tried with RAKUDO_MODULE_DEBUG=1?
[Coke] nine - updated gist with that output 15:54
C:\sandbox\rakudo>set RAKUDO_MODULE_DEBUG=1 15:55
C:\sandbox\rakudo>C:\sandbox\rakudo\rakudo-m.exe --ll-exception "C:\sandbox\rakudo\tools\build\install-core-dist.raku" "C:\raku\share\perl6\core" 1 RMD: R
equested for settings CORE.d
1 RMD: Loading settings CORE.d 1 RMD: Loading bytecod
e from CORE.d.setting.moarvm 1 RMD: going to load Perl6::BOOTSTRAP::v6d 1 RMD: Requested for set
tings CORE.c 1 RMD: Loading settings CORE.c
1 RMD: Loading bytecode from CORE.c.setting.moarvm
1 RMD: going to load Perl6::BOOTSTRAP::v6c 1 RMD: Settings
CORE.c loaded
1 RMD: Settings CORE.d loaded 1 RMD: Re
quested for settings CORE.d 1 RMD: Attempting 'lib' as a pragma 1 RMD: Successfully handled 'lib' as a pragma 1 RMD: Attempting 'CompUn
it::Repository::Staging' as a pragma 1 RMD: '
CompUnit::Repository::Staging' is not a valid pragma 1 RM
D: Attempting to load 'CompUnit::Repository::Staging' 1 RMD: Late loading 'CompUnit::Repository::Staging' 1 RMD:
try-load 33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5: C:\sandbox\rakudo\lib\CompUnit\Repository\Staging.rakumod 1 RMD: Trying to load 33A52
796DB3EBB40BEF94B7696A1B0AB7A29B5C5 from C:\sandbox\rakudo\lib\.precomp 1 RMD: Trying to loa
d 33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.repo-id from C:\sandbox\rakudo\lib\.precomp 1 RMD: Loading prec
ompiled C:\sandbox\rakudo\lib\.precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\33\33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5 1 RMD: Requested for settin
gs CORE.d 1 RMD: Performing imports for 'CompUnit::Repository::Staging' 1 RMD: Imports for 'CompUnit::Repository::Staging' done 1 RMD: Requested for settings CORE.d 1 RMD: Precompiling C:\r 15:56
aku\share\perl6\core\sources\10E86A71646D649AE0856ACE1737E1FFACC669D6 into C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\10\10E86A71646D649AE0856ACE1737E1FFACC669D6.bc (--ll-exception )
2 RMD: Requested for settings CORE.d 2 RMD: Loading settings CORE.d 2 RMD: Loading bytecode from CORE.d.setting.moarvm 2 RMD: going to load Pe
rl6::BOOTSTRAP::v6d 2 RMD: Requested for settings CORE.c 2 RMD: Loading se
ttings CORE.c 2 RMD: Loading bytecode from CORE.c.setting.moarvm 2 RMD: going to load Perl6::BOOTSTRAP::v6c 2 RMD: Settings CORE.c loaded 2 RMD: Settings CORE.d loaded 2 RMD: Attempting 'nqp' as
a pragma 2 RMD: Successfully handled 'nqp' as a pragma 1 RMD: Precomp
iled C:\raku\share\perl6\core\sources\10E86A71646D649AE0856ACE1737E1FFACC669D6 into C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\10\10E86A71646D649AE0856ACE1737E1FFACC669D6.bc 1 RMD: Writing dependencies and byte cod
e to C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\10\10E86A71646D649AE0856ACE1737E1FFACC669D6.tmp for source checksum: 730C935463B25D3071F7BF6251821D9504B56F99
1 RMD: Precompiling C:\raku\share\perl6\core\sources\70EBDA25F44EBFF8734F739F5779D64914083409 into C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\70\70EBDA25F44EBFF8734F739F5779D64914083409.bc (--ll-exception ) 2 RMD: Requested for settings CORE.d 2 RMD: Loading settings CORE.d 2 RMD: Loading bytecode from CORE.d.setting.moarvm 2 RMD: goin
g to load Perl6::BOOTSTRAP::v6d 2 RMD: Requested for settings CORE.c 2 RMD: Loading settings CORE.c 2 RMD: Loading bytecode from CORE.c.setting.moarvm 2 RMD: going to load Perl6::BOOTSTRAP::v6c 2 RMD: Settings CORE.c load
ed 2 RMD: Settings CORE.d loaded 1 RMD: Precompi
led C:\raku\share\perl6\core\sources\70EBDA25F44EBFF8734F739F5779D64914083409 into C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\70\70EBDA25F44EBFF8734F739F5779D64914083409.bc 1 RMD: Writing de
pendencies and byte code to C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\70\70EBDA25F44EBFF8734F739F5779D64914083409.tmp for source checksum: 708B4F48873D9AA1D048760B4B487CD8C7427E00 1 RMD: Precompiling C:\raku\share\perl6\core\sourc
es\09AD0895983003F8BD0D4FB6C3B0212C822A7FE8 into C:\raku\share\perl6\core\precomp\77F46E13F3D1DF44C7F80A37C6EF2AAED6FADE90\09\09AD0895983003F8BD0D4FB6C3B0212C822A7FE8.bc (--ll-exception ) 2 RMD: Requested for settings CORE.d 2 RMD: Loading s
ettings CORE.d 2 RMD: Loading by
tecode from CORE.d.setting.moarvm 2 RMD: going to load Perl6::BOOTSTRAP::v6d 2 RMD: Requested for settings CORE.c 2 RMD: Loading settings CORE
.c 2 RMD: Loading bytecode from CORE.c.setting.moarvm 2 RMD: 15:57
going to load Perl6::BOOTSTRAP::v6c 2 RMD: Setti
ngs CORE.c loaded 2 RMD: Settings CORE.d loaded 2 RMD: Attempting
'nqp' as a pragma 2 RMD: Successfully handled 'nqp' as a pragma 1 RMD: Pre
compiling C:\raku\share\perl6\core\sources\09AD0895983003F8BD0D4FB6C3B0212C822A7FE8 failed: 3221225512
at SETTING::src/core.c/Exception.pm6:62 (C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:throw) from SETTING::src/core.c/control.pm6:255
(C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:die) from
SETTING::src/core.c/CompUnit/PrecompilationRepository.pm6:456 (C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:precompile) from SETTING::src/core.c/CompUnit/Repository/Installation.pm6:301 (C:\sandbox\ra
kudo\blib/CORE.c.setting.moarvm:) from SETTING::src/core.c/CompUnit/Re
pository/Installation.pm6:291 (C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:)
15:57 brrt left
[Coke] from SETTING::src/core.c/CompUnit/Repository/Installation.pm6:274 (C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:) from SETTING::src/core.c/Lock.pm6:30 (C:\sandbox\ra 15:57
kudo\blib/CORE.c.setting.moarvm:protect) from
SETTING::src/core.c/CompUnit/Repository/Installation.pm6:177 (C:\sandbox\rakudo\blib/CORE.c.setting.moarvm:install) from C:\sandbox\raku
do\tools\build\install-core-dist.raku:46 (<ephemeral file>:<unit>)
from C:\sandbox\rakudo\tools\build\install-core-dist.raku:1 (<ephemeral file>:<unit-outer>) from gen\moar\stage
2\NQPHLL.nqp:1943 (C:\raku\share\nqp\lib/NQPHLL.moarvm:eval) from gen\moar\stage2\
NQPHLL.nqp:2148 (C:\raku\share\nqp\lib/NQPHLL.moarvm:evalfiles) from ge
n\moar\stage2\NQPHLL.nqp:2108 (C:\raku\share\nqp\lib/NQPHLL.moarvm:command_eval) from gen\moar\Compiler.nqp:111 (C:\sandb
ox\rakudo\blib/Perl6/Compiler.moarvm:command_eval) from gen\moar\stage2\NQPHLL.nqp:2033 (C:\raku\share\nqp\lib/NQPHLL.moarvm:command_line) from gen\moar\rakudo.nqp:127 (C:\sandbox\rakudo\rakudo.moarvm:MAIN) from gen\moar\rakudo.nqp:1 (C:\sandbox\rakudo\rakudo.moarvm:<mainline>) from
<unknown>:1 (C:\sandbox\rakudo\rakudo.moarvm:<main>)
OOF. sorry
was trying to paste the one interesting line. :| 15:58
nine The failing compilation is of MoarVM::Profiler 16:00
[Coke]: can you try something like ./rakudo-m lib/MoarVM/Profiler.rakumod? Maybe that will give us some more info 16:01
[Coke] seems to complete successfully 16:04
nine and ./rakudo-m -Ilib -e 'use MoarVM::Profiler'?
[Coke] precompile failed, error while compile -e 16:07
nine: gist.github.com/coke/7966426a43606...826948bba1 16:10
timo what if you just rakudo-m -Ilib lib/MoarVM/Profiler.rakumod?
nine What could that 3221225512 be? Seems to be always the same number 16:11
It's 0xC0000028 in hex.
[Coke] timo: that seems to complete 16:12
nine Ooooh: 16:14
nqp/MoarVM/3rdparty/libuv/src/win/winapi.h
836:# define STATUS_BAD_STACK ((NTSTATUS) 0xC0000028L)
Looking for the hexadecimal representation of that number was a really long shot but when you have nothing else at hand... :) 16:15
Now just grep through the OS source code to see in what situations this error gets reported 16:16
jnthnwrthngtn Hmm. bytepointer.com/resources/pietrek_...32_seh.htm 16:19
Does it go away with MVM_JIT_DISABLE=1?
Or at least maybe change? 16:22
[Coke] jnthnwrthngtn: yes! that gets .\rakudo-m -Ilib -e "use MoarVM::Profiler" to complete 16:28
Sorry. I should be trying that every time I have a weird error on win
(offline for a bit)
MasterDuke jnthnwrthngtn: question from #raku, "Is there a way to re-dispatch a method call to a role that the given class does? The method resolution order only includes parent classes, but not roles, so stuff like ā€˜callsameā€™ doesnā€™t work. Is anything like it available for roles, too?" 16:33
`RoleName::method-name()` gives `Could not find symbol '&method-name' in 'RoleName'`
jnthnwrthngtn self.RoleName::method-name() 16:34
It's correct that callsame doesn't visit them. Flattening composition means that a method in the class *replaces* the one that would be provided by the role.
MasterDuke ha, was just figured out over there
i didn't know it had to have `self.` before 16:36
jnthnwrthngtn Yes, otherwise it's not a method call. 16:41
MasterDuke ha, right 16:45
17:05 discord-raku-bot left
[Coke] back 17:06
17:06 discord-raku-bot joined
[Coke] guesses he can set MVM_JIT_DISABLE to get a working raku so he can run the scripts he needs for work, anyway. 17:06
17:14 patrickz left 17:17 patrickb joined
Geth MoarVM/debug-server-fixes: 86d0a05025 | (Jonathan Worthington)++ | src/debug/debugserver.c
Avoid duplicate response for suspend/resume all

The functions that do the suspension or resumption of all threads already communicate success or failure, so we don't need to do it from the message dispatch too.		 17:25
lizmat And yet another Rakudo Weekly News hits the Net: rakudoweekly.blog/2021/10/06/2021-40-its-here/ 17:37
jnthnwrthngtn lizmat++ # already read it, got advance notification because of the link to my 6guts post :) 17:38
lizmat :-)
jnthnwrthngtn bbl o/ 17:47
17:47 sena_kun left 18:02 reportable6 left 18:18 [Coke]_ joined 18:20 [Coke] left
nine Turns out, the segfaults in nativcall callback tests were not actually caused by the missing handling but by the NativeCall repr's copy_to neglecting to copy arg_info (and sym_name). Why this only occurs now is anyone's guess 18:45
Next failure mode: moar: src/core/callstack.c:425: handle_end_of_dispatch_record: Assertion `(char *)disp_record == (char *)tc->stack_top' failed. 18:46
MasterDuke maybe the copy_to problem is the cause of the failed nativecall tests on windows hat [Coke]_ saw? 18:48
18:57 brrt joined
brrt .tell MasterDuke I'm not seeing anything immediately obvious 19:00
tellable6 brrt, I'll pass your message to MasterDuke
MasterDuke hm, thanks for looking 19:01
jnthnwrthngtn nine: Is it in a callback test? It may be that the nested runloop messes stuff up 19:03
dinner &
Nicholas botherit - I wanted to ask a teasing question
in the new "disp all the things" paradigm, what was next after nativecall? Was it some thoughts about type coercions? 19:04
19:22 brrt left
nine jnthnwrthngtn: yes, it is 19:28
jnthnwrthngtn: and yes, it does 19:35
Just restoring tc->stack_top to its previous state after the nested runloop gets me to the end of the test file with just 3 out of 11 failed. But of course that's a rather crude way. 19:40
20:03 reportable6 joined
jnthnwrthngtn nine: Eventually we should probably have a callstack record type for "edge of a nested runloop" 20:31
(However, that may not make sense until the larger restructuring of call/return I'm plotting) 20:32
20:32 [Coke]_ is now known as [Coke]
jnthnwrthngtn Nicholas: Integrating the specialization selection with dispatch programs is probably one more useful unification. Not quite sure how to achieve it yet. 20:33
20:44 patrickb left
nine jnthnwrthngtn: so do you consider restoring tc->stack_top to be a valid workaround till that restructuring? 20:45
jnthnwrthngtn nine: If it suffices, I think so, yes
I think it's relatively unlikely callbacks would work at all if this was the problem 20:46
[Coke] with JIT disabled I was able to install raku. But then installing zef dies. 20:54
ah. also getting lots of 'nmake test' failures, will post. 20:56
I do apologize because I feel like 50% of these are "will did something dumb." :)
21:44 linkable6 left, evalable6 left, linkable6 joined 21:45 evalable6 joined
Geth MoarVM: 627c92ccb0 | (Jonathan Worthington)++ | 3 files
Add [mono|poly|mega]morphic site counts in profile

Including which megamorphic ones ended up blowing the cache size limit, indicating there was no megamorphic handler installed to cope with the situation.		 22:24
timo oh, cool 22:25
jnthnwrthngtn Wonder how best to do this in the SQL 22:30
timo can just add a new table that has the values indexed to the id from the call table 22:31
the nqp side of the code can check if any node has the mono/poly/etc keys and emit the table definition a little later or whatever 22:32
oh, wait, this isn't about call, this is about routine, isn't it 22:33
jnthnwrthngtn oh hmm...yes, it's about routine, but I've emitted it per call node, which is...
timo not optimal, eah
jnthnwrthngtn grmbl
timo shouldn't be difficult to move to the right spot
should go in the "get routine id" function that creates the routine hash if necessary 22:34
jnthnwrthngtn Is there a separate notion of routines vs calls in the data structure MoarVM builds up?
timo oh 22:35
i must have confused it with the implementation of the heap dump
every call has the name and line number and filename in the node as strings and ints 22:38
jnthnwrthngtn Yup
Though it seems there is a unique ID per static frame too
So I can clear it up into a per-routine thing on the NQP side and have a separate table without the repetition. 22:39
timo OTOH since the contents of a cache change over time, there is a little bit of a reason to have it in the call graph
jnthnwrthngtn But I don't log any info 22:40
I add the end state of the inline caches
timo ah, OK
jnthnwrthngtn But yeah, this is...rather inefficient.
timo time for a full rewrite
jnthnwrthngtn Although doing better probably means a larger reorganization.
timo i also notice we box_s the filenames over and over, so that's some memory definitely wasted there 22:41
but only RAM usage at the time you pull out the profile data
jnthnwrthngtn The profiler as it is today is mostly quite recognizable from the Conference Driven Development implementation I wrote years ago :) 22:42
timo that's because you did it good :D
jnthnwrthngtn I guess we have a decent amount of freedom if we want to tidy up, because the only thing that we absolutely cannot change without causing big problems is the SQL 22:44
Which is already sensible enough to have a separate routines table
MasterDuke when i started using raku (perl 6 at the time) i thought one of the coolest features was that it came with a profiler
timo having it output a html file that just gives you all the data is damn nice 22:45
22:45 linkable6 left, evalable6 left
jnthnwrthngtn While the `--profile` at the command line spitting out a HTML single page app only scales so far, it's delightfully easy to use 22:45
timo sadly, browsers don't like when html files go past a couple megabytes
jnthnwrthngtn I suspect we could get it to go a little further with a more compact representation. 22:46
MasterDuke i think it's really just the size of the json that's a problem, not of the page in total
22:46 evalable6 joined
jnthnwrthngtn For sure, though we can probably get it more compact too 22:47
MasterDuke wins all around
timo toss out key/value, replace it with arrays with fixed indices, for example
jnthnwrthngtn Things like this, yes
timo oh, can possibly gzip or whatever it before writing to the html file, then ungzip it on the fly in javascript
jnthnwrthngtn o.O
timo may have to base64 it in the middle tho 22:48
jnthnwrthngtn Oh, talking of that...about heap profiles: what is used to compress them, and what is its license?
MasterDuke probably even faster to stick a webasm zstd implementation in and use that
timo zstd, the library has a compatible license, i checked
a moment please
3 clause bsd 22:49
jnthnwrthngtn OK, so there'd be no problem with us having it as a 3rdparty/ dep to ensure it's present on systems that don't have a package for it?
timo that would be all right i think
MasterDuke the linux kernel is just about to upgrade their zstd implementation, it's something like 4 years out of date
jnthnwrthngtn I'm looking at Comma heap profile support and it's a bit annoying that many folks will have MoarVM versions that don't support version 3 of the heap profile (I think it's version 3) 22:50
timo do you think it's worth dynamically loading libzstd when it's first needed, so that moarvm's memory footprint doesn't increase much?
i would be in favor of making zstd available for all users
jnthnwrthngtn Is it such a big library? 22:51
Ah, maybe some big tables that drive compression?
timo libzstd.so.1.5.0 is 988K big on my system
jnthnwrthngtn Wowser, that's more than I'd expect
timo i don't think i have bloaty installed
jnthnwrthngtn Presumably any heap allocations can be delayed until it's initialized and we can do that lazily, and we can leave demand paging and memory mapping to keep the cost down 22:52
timo the dynamic loader is pretty fast, that's true 22:53
MasterDuke timo: gist.github.com/MasterDuke17/7a1aa...27f2bf922d
jnthnwrthngtn Well, if I understand that output correctly, not a bunch of big static tables then 22:54
timo [13] .text PROGBITS 00003ec0 003ec0 0dfbd8 00 AX 0 0 16
[15] .rodata PROGBITS 000e4000 0e4000 005745 00 A 0 0 32
[17] .eh_frame PROGBITS 000ea4d4 0ea4d4 009f24 00 A 0 0 4
these are the big sections
jnthnwrthngtn Yeah but 22:55
90.5% 975Ki 91.1% 975Ki .text
That dominates
timo oof.
jnthnwrthngtn And that's the machine code section, iiuc?
timo yeah
could potentially build a libzstd that can only compress not decompress
no clue how much that can save
jnthnwrthngtn I've no idea what eh_frame is, I figure rodata is where any of the tables I was imagining would be 22:56
timo exception handlers? 22:57
jnthnwrthngtn Oh...but it looks like it's in C? Or is it so a C++ program using it can throw over it? 22:58
At least if number of files is anything to go on, compress is heavier than decompress
MasterDuke github.com/facebook/zstd/blob/dev/...ING.md#c90 22:59
timo Languages
C 84.2%
C++ 6.4%
Shell 2.9%
Python 2.2%
Makefile 2.0%
CMake 0.7%
Other 1.6%
presumably a cpp compatible interface in the same .so file 23:00
jnthnwrthngtn ah, maybe there's a configure option to avoid that since we won't need it 23:01
For when we build it ourselves at least
timo there's also the zlib wrapper which lets you drop in zstd as replacement and switch back and forth with a define
no clue how much that contributes
the functionality to support and create pre-trained shared dictionaries may be useless to us as well 23:02
otherwise, static linking and dead code elimination could be an option 23:04
jnthnwrthngtn I guess the other view on this is that wider access to heap snapshots may well lead to more memory being saved than zstd costs :) 23:06
timo :D
MasterDuke there doesn't really appear to be any configuring options when building it (at least as far as leaving out features and such) 23:11
23:46 linkable6 joined

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!