Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:09 reportable6 left 00:12 reportable6 joined 00:52 evalable6 joined 01:35 frost joined 01:42 frost left 02:27 frost joined 02:39 frost left 03:10 frost joined 03:13 discord-raku-bot left, discord-raku-bot joined 03:17 discord-raku-bot left, discord-raku-bot joined 03:20 frost left 04:10 frost_ joined 04:14 frost_ left 04:16 frost joined 04:20 frost left 04:23 frost joined 04:30 frost left 05:10 frost joined 05:25 Kaipi joined 05:35 discord-raku-bot left, kjp left 05:37 RakuIRCLogger__ joined 05:38 RakuIRCLogger left 05:40 discord-raku-bot joined, kjp joined, Altai-man joined, TempIRCLogger joined, japhb joined, Geth joined 06:08 reportable6 left 06:14 frost left
Nicholas good *, #moarvm 06:41
07:10 reportable6 joined
nine Oh is it * again? 07:18
Nicholas the light is on in the big blue room - it certainly looks that way 07:22
nine Isn't much light though, so I remain skeptical
08:03 patrickb joined, patrickb left 08:12 patrickb joined
Nicholas nine: the window in this room faces (roughly) south and it was fairly blue that way. But going upstairs to get coffee, I can see a thick wall of grey in your direction. 08:24
Looks pretty solid (at least for clouds). So if I extrapolate wildly ("this is what we do in the Internet, right?") that it's 200km or so thick, that will explain everything? 08:25
fresh pack of coffee. smells good...
nine The first whiff when you open a pack, that's perfection 08:40
MasterDuke i love the little one-way valve they have so you can squeeze a closed pack and still get that smell 08:41
Voldenet Who said it's one way – when you smell coffee the coffee also smells you. 08:57
Whoever smells coffee should see to it that he does not become coffee in the process.
this quote makes less sense in the context than I initially thought, hm 08:58
Nicholas this feels like a feed line, that leads to "you are typing this before you have had coffee?" 09:00
Voldenet Yes 09:01
Geth MoarVM/master: 9 commits pushed by (Nicholas Clark)++, (Daniel Green)++ 09:15
lizmat good *! 09:16
is this bumpable? nine: or should I wait until you've bumped?
Nicholas lets wait until the CI runs...
MasterDuke oooh, maybe now it's time to experiment with removing the fsa completely... 09:20
Nicholas it's also not clear whether 09:26
oops
whether mi_good_size() is useful anywhere microsoft.github.io/mimalloc/group...47e921bd47
ie - is there code that allocates a buffer of size $whatever, and records that $whatever with the pointer to the buffer, so that later code can grow the buffer if needed 09:27
nine arrays
Nicholas Ah OK good
anyway, IIRC macOS system malloc has something similar
and it's trvial not to be tied to mimalloc with this - the fallback wrapper is #define MVM_good_size(foo) (foo) 09:28
nine [ 56s] src/moar.h:29:10: fatal error: mimalloc.h: No such file or directory 09:32
[ 56s] 29 | #include <mimalloc.h>
build.opensuse.org/package/live_bu...ARM/armv7l
Same here: build.opensuse.org/package/live_bu...RM/aarch64
Nicholas it's in .gitmodules
nine My script does git pull ; git submodule update 09:34
Nicholas I think it might need a `git submodule init` too. *But* I don't know this stuff well. And I forget what happened with ryu 09:35
nine git submodule update has an optional --init flag, but I can't find docs on what it does... 09:37
Voldenet you need git submodule update –recursive
nine Ah, it's documented in git submodule init 09:39
I changed it to git -C MoarVM submodule update --init --recursive just to be sure. We'll see how it goes
Voldenet git has submodule.recurse setting which makes pulls a lot easier 09:41
MasterDuke oh, i realized this the other day but forgot to comment, we probably want something like `#ifdef MIMALLOC #define MVM_malloc_trim mi_collect #else #define MVM_malloc_trim malloc_trim #endif` 09:44
Geth MoarVM: MasterDuke17++ created pull request #1641:
Make value_desc_cont_atomic_store() static		 09:59
MoarVM: 1351cfe6fd | (Daniel Green)++ | src/6model/containers.c
Make value_desc_cont_atomic_store() static

Looks like it was just an accidental omission when it was added.		 10:00
MoarVM: fcbbfc660f | MasterDuke17++ (committed using GitHub Web editor) | src/6model/containers.c
Merge pull request #1641 from MasterDuke17/make_value_desc_cont_atomic_store_static
10:22 frost joined
MasterDuke ugh, i don't think we can easily switch MVM_recalloc to use mi_recalloc. we currently pass the old size and the new size to MVM_recalloc, and then manually memset the difference. but mi_recalloc takes the new count and new size (like calloc) 10:29
so if we change MVM_recalloc to take count and size then where we don't use mimalloc we can't know what to memset 10:30
moon-child why not just pass mi_recalloc a count of n and size of 1 (or vice versa)? 10:31
MasterDuke well, i was just thinking about that 10:32
but i wonder if that'll throw off any expectations/optimizations in the implementation of mi_recalloc. i.e., yeah it should work correctly, but will performance be negatively effected? 10:33
moon-child literally the only reason calloc takes separately a count and a size is so it can do overflow checking 10:34
there's no reason to do more than that
github.com/microsoft/mimalloc/blob...loc.c#L664 10:35
10:35 frost left
moon-child
.oO(could also just call rezalloc)
 10:35
MasterDuke ah, nice. i wondered if perhaps it did something with it's various heaps and bin sizes, but looks like not 10:37
jnthnwrthngtn moarning o/ 10:40
Nicholas \o 10:43
10:53 frost joined
MasterDuke huh.gist.github.com/MasterDuke17/b8d65...16912c6af4 causes an insta-MVM_panic in the nqp build 10:54
which i guess means somewhere we're recallocing memory that wasn't originally calloced 10:55
now how to find where... 10:56
Nicholas rr?
this seems like a "wind time backwards" problem
MasterDuke oh, but github.com/microsoft/mimalloc/issues/63 suggest mi_rezalloc is going away... 10:58
well, i guess no real harm in leaving things as they are right now, the mimalloc implementation pretty much does just what we do (i.e., memset to 0 the difference) 11:04
timo it's wind time! except backwards, so it would be dniw 11:06
11:39 sena_kun joined 11:55 frost left 12:07 reportable6 left
MasterDuke iirc,the fsa provided two benefits. first was faster allocation for small sizes, which is apparently not so useful now that we're using mimalloc. but the second was the *_at_safepoint mechanism, which was part of the point of using the fsa for things like the body (actual slots + metadata) of arrays, so that modifying them from separate threads 12:14
would cause things like an MVM_panic instead of a segv
so simply removing the fsa entirely will regress that second benefit. instead, should create MVM_*_at_safepoint functions that just use regular MVM_* alloc functions instead of MVM_fixed_size_* alloc functions 12:16
does this sound correct? 12:18
12:19 frost joined
nine yes 12:25
12:26 frost left
Nicholas seems correct to me. Also, as the FSA already has to handle pointers to allocations larger than its largest bucket, the code must exist already 12:40
12:42 frost joined
Geth MoarVM: nwc10++ created pull request #1642:
update-submodules.pl should handle linked worktrees created by git-worktree		 12:55
13:11 sena_kun_ joined 13:13 sena_kun left 14:11 reportable6 joined 14:52 frost left 14:55 discord-raku-bot left, discord-raku-bot joined 15:19 RakuIRCLogger__ left, lizmat left, Geth left 15:20 TempIRCLogger left 15:29 TempIRCLogger joined 15:30 lizmat joined, RakuIRCLogger joined 15:34 RakuIRCLogger left 15:35 RakuIRCLogger joined 15:46 TempIRCLogger left, RakuIRCLogger left 15:48 lizmat left 15:49 TempIRCLogger joined 16:03 Geth joined
lizmat hmmm... looks like my ISP had 2 outages: one 11 mins, and one 5 mins 16:06
fortunately, the loggers recovered ok 16:07
but Geth needed a manual restart somehow
16:11 discord-raku-bot left 16:12 discord-raku-bot joined, cognominal joined 17:10 dogbert17 left 17:20 dogbert17 joined 17:54 [Coke] left 17:58 [Coke] joined 18:08 reportable6 left 18:09 reportable6 joined 18:15 Geth left, Geth joined 18:26 nebuchad` joined 18:27 SmokeMachine_ joined, childlikempress joined, leont_ joined 18:29 jnthn joined, sena_kun_ left 18:30 nine_ joined 18:32 moon-chi- joined 18:34 childlikempress left, nine left, moon-child left, SmokeMachine left, nebuchadnezzar left, leont left, jnthnwrthngtn left, rba left, cognominal left, RakuIRCLogger left, sena_kun_ left, kjp left, Altai-man left, japhb left, patrickb left, MasterDuke left, bartolin left, leedo left, Techcable left, ugexe left, camelia left, SmokeMachine_ is now known as SmokeMachine, rba_ is now known as rba, leont_ is now known as leont 18:35 nine left, moon-child left, SmokeMachine left, nebuchadnezzar left, leont left, jnthnwrthngtn left, rba left, Geth left, dogbert17 left, lizmat left, Voldenet left, gfldex left, AlexDaniel left, JRaspass left, jdv left, Nicholas left, samcv left, timo left, ilogger2 left, harrow left, samebchase left, TempIRCLogger left, psydroid left, rypervenche left, tbrowder left, Util_ left, reportable6 left, [Coke] left, discord-raku-bot left, Kaipi left, evalable6 left, linkable6 left, squashable6 left, sourceable6 left, quotable6 left, statisfiable6 left, unicodable6 left, notable6 left, committable6 left, releasable6 left, tellable6 left, bloatable6 left, benchable6 left, bisectable6 left, nativecallable6 left, coverable6 left, shareable6 left, greppable6 left, codesections left, vrurg left 18:38 discord-raku-bot left, discord-raku-bot joined, nebuchad` joined, cognominal joined, RakuIRCLogger joined, patrickb joined, japhb joined, Altai-man joined, kjp joined, MasterDuke joined, Techcable joined, camelia joined, bartolin joined, leedo joined, ugexe joined 18:39 moon-chi- joined, nine_ joined, jnthn joined, leont joined, SmokeMachine joined, rba joined, Geth joined, reportable6 joined, [Coke] joined, dogbert17 joined, TempIRCLogger joined, lizmat joined, Kaipi joined, evalable6 joined, linkable6 joined, squashable6 joined, sourceable6 joined, quotable6 joined, statisfiable6 joined, unicodable6 joined, notable6 joined, committable6 joined, releasable6 joined, tellable6 joined, bloatable6 joined, benchable6 joined, bisectable6 joined, nativecallable6 joined, coverable6 joined, shareable6 joined, greppable6 joined, codesections joined, vrurg joined, psydroid joined, AlexDaniel joined, gfldex joined, Voldenet joined, jdv joined, JRaspass joined, samebchase joined, harrow joined, timo joined, samcv joined, Nicholas joined, ilogger2 joined, Util_ joined, tbrowder joined, rypervenche joined 18:40 discord-raku-bot joined 19:42 bloatable6 left, nativecallable6 left, coverable6 left, statisfiable6 left, notable6 left, sourceable6 left, reportable6 left, quotable6 left, greppable6 left, linkable6 left, bisectable6 left, evalable6 left, releasable6 left, benchable6 left, tellable6 left, committable6 left, shareable6 left, squashable6 left, unicodable6 left, evalable6 joined 19:43 greppable6 joined, benchable6 joined, committable6 joined, notable6 joined, nativecallable6 joined
Nicholas OK, naughty denizens of #moarvm, struct MVMRegister *might* need 8-byte alignment on 32 bit systems... 19:55
(this is progress)
19:55 moon-chi- is now known as moon-child 20:01 discord-raku-bot left, discord-raku-bot joined
Nicholas sparc64++ # the best bastard out there for checking alignment 20:05
sparc64 hardware in 32 bit mode is somewhat more fun than sparc64 hardware in 64 bit mode 20:06
because the compiler *knows*
OK, culprit is: 20:35
tc->stack_top = allocate_record(tc, MVM_CALLSTACK_RECORD_DISPATCH_RUN,
sizeof(MVMCallStackDispatchRecord) + num_temps * sizeof(MVMRegister));
figuring an elegant solution... 20:36
20:38 nine_ is now known as nine
nine Could that cause those strange hangs I see on aarch64 and armv7l? 20:40
Nicholas unlikely. it's an failing-to-be-64-bit-aligned issue, which seems only to matter on sparc 20:41
(and probably ia64, which I don't have useful access to, and maybe parisc64 and alpha. I raelised about 15 minutes ago that there's a connection between all of those)
and I can't replicate your hang, but I don't know how the system I tried it on differs 20:42
nine They are kinda random
20:42 bloatable6 joined
Nicholas so maybe I need a loop... 20:43
20:43 coverable6 joined, statisfiable6 joined 20:44 quotable6 joined, unicodable6 joined, bisectable6 joined
Nicholas jnthn: on line 302 of src/core/callstack.c, why is the allocation calculated as sizeof(MVMCallStackDispatchRecord) and not sizeof(MVMCallStackDispatchRun) ? 20:58
nine Smells fishy 21:07
Nicholas it's (relatively) safe - allocated sizeof(...) is 200, used sizeof(...) is 76
but does seem wrong
nine Probably costs a tiny bit of performance 21:09
MasterDuke ugh. just removed the fsa, but get a segfault pretty soon in the nqp build 21:31
oh, i bet need to rebootstrap because i removed stuff from MVMInstance 21:37
jnthn Nicholas: Sounds like copy-pasta 21:39
Oh or 21:40
Hm, I notice after some reconnect my nick as changed :)
21:40 jnthn is now known as jnthnwrthngtn
jnthnwrthngtn Alas, that one's still reg'd 21:40
Anyway, if none of the dispatch programs we run work out we'd need to then record a new one, so it's possible the thinking was to permit reuse of the record and then events overtook it. I don't really remember. 21:41
(I know how it ended up is discarding the run record and making a record record)
But probably it was copy pasta 21:42
21:42 tellable6 joined 21:43 sourceable6 joined, reportable6 joined 21:44 shareable6 joined 21:45 releasable6 joined
MasterDuke does gist.github.com/MasterDuke17/d1c26...32c4cbffbf look like a correct implementation of MVM_realloc_at_safepoint()? 21:58
jnthnwrthngtn No. You can't actually implement it in terms of realloc 22:03
MasterDuke oh?
has to be a new alloc? 22:04
jnthnwrthngtn Yes, otherwise there's no previous alloc to defer the deallocation of until a safepoint
Yes, otherwise there's no previous alloc to defer the deallocation of until a safepoint 22:05
The only optimization available is to do nothing is new_size == old_size (or under the FSA, if they would go into the same bin)
Ah, the other optimization is if minmalloc exposes something where you can say "do the realloc but only if you don't change the pointer" 22:07
(that is, it can enlarge it because there's already free memory after the block)
I've no idea if it has such a feature
MasterDuke yeah, i think they have mi_expand or something like that
jnthnwrthngtn OK, that may allow safety + more efficiency
The rule is that any memory reads based on the pointer passed into realloc up to the extent of the original size must continue to work until the safepoint 22:08
(Sorry if that's not very clearly staged, I'm a little tired...)
heh, stated. Point proved :D
MasterDuke huh. nqp built ok and passed `make m-test`, but the rakudo build just died in stage parse with `This representation (P6int) cannot unbox to a native string (for type BOOTInt)`
thanks, that's all helpful 22:09
jnthnwrthngtn I can imagine NQP building with this wrong given safepoint-y stuff only tends to come into play when there's multiple threads at work 22:10
MasterDuke rakudo just built (this is all with the fsa 99.9% removed) 22:15
jnthnwrthngtn Will be interesting to see any timing results :)
MasterDuke stage parse looked about the same
jnthnwrthngtn The same but with less code we have to maintain is still a win, I guess. 22:16
MasterDuke looks like ~500 fewer lines 22:18
hm, bunch of spectest fails though 22:23
23:44 linkable6 joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!