Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
Nicholas good *, #moarvm 06:41
nine Oh is it * again? 07:18
Nicholas the light is on in the big blue room - it certainly looks that way 07:22
nine Isn't much light though, so I remain skeptical
Nicholas nine: the window in this room faces (roughly) south and it was fairly blue that way. But going upstairs to get coffee, I can see a thick wall of grey in your direction. 08:24
Looks pretty solid (at least for clouds). So if I extrapolate wildly ("this is what we do in the Internet, right?") that it's 200km or so thick, that will explain everything? 08:25
fresh pack of coffee. smells good...
nine The first whiff when you open a pack, that's perfection 08:40
MasterDuke i love the little one-way valve they have so you can squeeze a closed pack and still get that smell 08:41
Voldenet Who said it's one way – when you smell coffee the coffee also smells you. 08:57
Whoever smells coffee should see to it that he does not become coffee in the process.
this quote makes less sense in the context than I initially thought, hm 08:58
Nicholas this feels like a feed line, that leads to "you are typing this before you have had coffee?" 09:00
Voldenet Yes 09:01
Geth MoarVM/master: 9 commits pushed by (Nicholas Clark)++, (Daniel Green)++ 09:15
lizmat good *! 09:16
is this bumpable? nine: or should I wait until you've bumped?
Nicholas lets wait until the CI runs...
MasterDuke oooh, maybe now it's time to experiment with removing the fsa completely... 09:20
Nicholas it's also not clear whether 09:26
oops
whether mi_good_size() is useful anywhere microsoft.github.io/mimalloc/group...47e921bd47
ie - is there code that allocates a buffer of size $whatever, and records that $whatever with the pointer to the buffer, so that later code can grow the buffer if needed 09:27
nine arrays
Nicholas Ah OK good
anyway, IIRC macOS system malloc has something similar
and it's trvial not to be tied to mimalloc with this - the fallback wrapper is #define MVM_good_size(foo) (foo) 09:28
nine [ 56s] src/moar.h:29:10: fatal error: mimalloc.h: No such file or directory 09:32
[ 56s] 29 | #include <mimalloc.h>
build.opensuse.org/package/live_bu...ARM/armv7l
Same here: build.opensuse.org/package/live_bu...RM/aarch64
Nicholas it's in .gitmodules
nine My script does git pull ; git submodule update 09:34
Nicholas I think it might need a `git submodule init` too. *But* I don't know this stuff well. And I forget what happened with ryu 09:35
nine git submodule update has an optional --init flag, but I can't find docs on what it does... 09:37
Voldenet you need git submodule update –recursive
nine Ah, it's documented in git submodule init 09:39
I changed it to git -C MoarVM submodule update --init --recursive just to be sure. We'll see how it goes
Voldenet git has submodule.recurse setting which makes pulls a lot easier 09:41
MasterDuke oh, i realized this the other day but forgot to comment, we probably want something like `#ifdef MIMALLOC #define MVM_malloc_trim mi_collect #else #define MVM_malloc_trim malloc_trim #endif` 09:44
Geth MoarVM: MasterDuke17++ created pull request #1641:
Make value_desc_cont_atomic_store() static		 09:59
MoarVM: 1351cfe6fd | (Daniel Green)++ | src/6model/containers.c
Make value_desc_cont_atomic_store() static

Looks like it was just an accidental omission when it was added.		 10:00
MoarVM: fcbbfc660f | MasterDuke17++ (committed using GitHub Web editor) | src/6model/containers.c
Merge pull request #1641 from MasterDuke17/make_value_desc_cont_atomic_store_static
MasterDuke ugh, i don't think we can easily switch MVM_recalloc to use mi_recalloc. we currently pass the old size and the new size to MVM_recalloc, and then manually memset the difference. but mi_recalloc takes the new count and new size (like calloc) 10:29
so if we change MVM_recalloc to take count and size then where we don't use mimalloc we can't know what to memset 10:30
moon-child why not just pass mi_recalloc a count of n and size of 1 (or vice versa)? 10:31
MasterDuke well, i was just thinking about that 10:32
but i wonder if that'll throw off any expectations/optimizations in the implementation of mi_recalloc. i.e., yeah it should work correctly, but will performance be negatively effected? 10:33
moon-child literally the only reason calloc takes separately a count and a size is so it can do overflow checking 10:34
there's no reason to do more than that
github.com/microsoft/mimalloc/blob...loc.c#L664 10:35
moon-child
.oO(could also just call rezalloc)
 10:35
MasterDuke ah, nice. i wondered if perhaps it did something with it's various heaps and bin sizes, but looks like not 10:37
jnthnwrthngtn moarning o/ 10:40
Nicholas \o 10:43
MasterDuke huh.gist.github.com/MasterDuke17/b8d65...16912c6af4 causes an insta-MVM_panic in the nqp build 10:54
which i guess means somewhere we're recallocing memory that wasn't originally calloced 10:55
now how to find where... 10:56
Nicholas rr?
this seems like a "wind time backwards" problem
MasterDuke oh, but github.com/microsoft/mimalloc/issues/63 suggest mi_rezalloc is going away... 10:58
well, i guess no real harm in leaving things as they are right now, the mimalloc implementation pretty much does just what we do (i.e., memset to 0 the difference) 11:04
timo it's wind time! except backwards, so it would be dniw 11:06
MasterDuke iirc,the fsa provided two benefits. first was faster allocation for small sizes, which is apparently not so useful now that we're using mimalloc. but the second was the *_at_safepoint mechanism, which was part of the point of using the fsa for things like the body (actual slots + metadata) of arrays, so that modifying them from separate threads 12:14
would cause things like an MVM_panic instead of a segv
so simply removing the fsa entirely will regress that second benefit. instead, should create MVM_*_at_safepoint functions that just use regular MVM_* alloc functions instead of MVM_fixed_size_* alloc functions 12:16
does this sound correct? 12:18
nine yes 12:25
Nicholas seems correct to me. Also, as the FSA already has to handle pointers to allocations larger than its largest bucket, the code must exist already 12:40
Geth MoarVM: nwc10++ created pull request #1642:
update-submodules.pl should handle linked worktrees created by git-worktree		 12:55
lizmat hmmm... looks like my ISP had 2 outages: one 11 mins, and one 5 mins 16:06
fortunately, the loggers recovered ok 16:07
but Geth needed a manual restart somehow
Nicholas OK, naughty denizens of #moarvm, struct MVMRegister *might* need 8-byte alignment on 32 bit systems... 19:55
(this is progress)
Nicholas sparc64++ # the best bastard out there for checking alignment 20:05
sparc64 hardware in 32 bit mode is somewhat more fun than sparc64 hardware in 64 bit mode 20:06
because the compiler *knows*
OK, culprit is: 20:35
tc->stack_top = allocate_record(tc, MVM_CALLSTACK_RECORD_DISPATCH_RUN,
sizeof(MVMCallStackDispatchRecord) + num_temps * sizeof(MVMRegister));
figuring an elegant solution... 20:36
nine Could that cause those strange hangs I see on aarch64 and armv7l? 20:40
Nicholas unlikely. it's an failing-to-be-64-bit-aligned issue, which seems only to matter on sparc 20:41
(and probably ia64, which I don't have useful access to, and maybe parisc64 and alpha. I raelised about 15 minutes ago that there's a connection between all of those)
and I can't replicate your hang, but I don't know how the system I tried it on differs 20:42
nine They are kinda random
Nicholas so maybe I need a loop... 20:43
Nicholas jnthn: on line 302 of src/core/callstack.c, why is the allocation calculated as sizeof(MVMCallStackDispatchRecord) and not sizeof(MVMCallStackDispatchRun) ? 20:58
nine Smells fishy 21:07
Nicholas it's (relatively) safe - allocated sizeof(...) is 200, used sizeof(...) is 76
but does seem wrong
nine Probably costs a tiny bit of performance 21:09
MasterDuke ugh. just removed the fsa, but get a segfault pretty soon in the nqp build 21:31
oh, i bet need to rebootstrap because i removed stuff from MVMInstance 21:37
jnthn Nicholas: Sounds like copy-pasta 21:39
Oh or 21:40
Hm, I notice after some reconnect my nick as changed :)
jnthnwrthngtn Alas, that one's still reg'd 21:40
Anyway, if none of the dispatch programs we run work out we'd need to then record a new one, so it's possible the thinking was to permit reuse of the record and then events overtook it. I don't really remember. 21:41
(I know how it ended up is discarding the run record and making a record record)
But probably it was copy pasta 21:42
MasterDuke does gist.github.com/MasterDuke17/d1c26...32c4cbffbf look like a correct implementation of MVM_realloc_at_safepoint()? 21:58
jnthnwrthngtn No. You can't actually implement it in terms of realloc 22:03
MasterDuke oh?
has to be a new alloc? 22:04
jnthnwrthngtn Yes, otherwise there's no previous alloc to defer the deallocation of until a safepoint
Yes, otherwise there's no previous alloc to defer the deallocation of until a safepoint 22:05
The only optimization available is to do nothing is new_size == old_size (or under the FSA, if they would go into the same bin)
Ah, the other optimization is if minmalloc exposes something where you can say "do the realloc but only if you don't change the pointer" 22:07
(that is, it can enlarge it because there's already free memory after the block)
I've no idea if it has such a feature
MasterDuke yeah, i think they have mi_expand or something like that
jnthnwrthngtn OK, that may allow safety + more efficiency
The rule is that any memory reads based on the pointer passed into realloc up to the extent of the original size must continue to work until the safepoint 22:08
(Sorry if that's not very clearly staged, I'm a little tired...)
heh, stated. Point proved :D
MasterDuke huh. nqp built ok and passed `make m-test`, but the rakudo build just died in stage parse with `This representation (P6int) cannot unbox to a native string (for type BOOTInt)`
thanks, that's all helpful 22:09
jnthnwrthngtn I can imagine NQP building with this wrong given safepoint-y stuff only tends to come into play when there's multiple threads at work 22:10
MasterDuke rakudo just built (this is all with the fsa 99.9% removed) 22:15
jnthnwrthngtn Will be interesting to see any timing results :)
MasterDuke stage parse looked about the same
jnthnwrthngtn The same but with less code we have to maintain is still a win, I guess. 22:16
MasterDuke looks like ~500 fewer lines 22:18
hm, bunch of spectest fails though 22:23

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!