Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:31 evalable6 joined 01:03 reportable6 joined 02:24 quotable6 left, tellable6 left, unicodable6 left, statisfiable6 left, linkable6 left, squashable6 left, greppable6 left, notable6 left, sourceable6 left, coverable6 left, bloatable6 left, nativecallable6 left, benchable6 left, bisectable6 left, committable6 left, reportable6 left, releasable6 left, shareable6 left, releasable6 joined, unicodable6 joined 02:25 committable6 joined, coverable6 joined, bisectable6 joined 02:26 statisfiable6 joined, greppable6 joined 02:27 reportable6 joined, squashable6 joined, shareable6 joined
japhb starts a really slow test, goes and eats dinner, comes back to "MoarVM panic: Tried to garbage-collect a locked mutex" ... GAH 02:46
Sigh
03:25 notable6 joined 04:24 bloatable6 joined 04:25 quotable6 joined, tellable6 joined 04:26 benchable6 joined 05:27 linkable6 joined 06:02 reportable6 left 06:05 reportable6 joined
MasterDuke japhb: i've never seen that panic before. can you run it in gdb with a breakpoint on MVM_panic? 06:39
06:42 gfldex left 06:44 gfldex joined
nine Was --full-cleanup involved? 06:46
moon-child wait, what's wrong with collecting a locked 07:13
it's not something that should come up, given proper use of structured concurrency, but it doesn't seem semantically incorrect?
*collecting a locked _mutex_
nine moon-child: may be OK semantics, but still can't be done. From PTHREAD_MUTEX_DESTROY(3P): Attempting to destroy a locked mutex, or a mutex that another thread is attempting to lock, or a mutex that is being used in a pthread_cond_timedwait() or pthread_cond_wait() call by another thread, results in undefined behavior. 07:16
moon-child well sure, it's the garbage collector's job to unlock the mutex before destroying it. Just as it's its job to close a file if you leave it open and then drop all references to it 07:18
(even though, again, properly written code is unlikely to leave files in a state where such a thing can happen) 07:19
nine That would require the GC to run on the same thread that's holding the lock on that mutex. 07:20
moon-child huh...for some reason I thought you were allowed to unlock mutexes of other threads? 07:22
I remember somebody's saying they had made a trivial thread lib wrapper that checked, when unlocking, whether the unlocking thread was the thread that had originally locked and complained if not. I assumed that was because the behaviour otherwise was to just unlock the mutex 07:23
nine No, that either results in undefined behavior or a straight error, depending on the type of mutex
moon-child yeah, I see that now
07:26 sourceable6 joined, nativecallable6 joined
nine Well I asked about --full-cleanup because there the main source of such locked mutexes is threads that are still running when we're trying to shut down. For that I have a candidate fix: github.com/rakudo/rakudo/commit/8d...65155a13e0 07:27
08:42 Kaipi left
lizmat commented on that commit 09:09
MasterDuke hm. just tried to add two new expression operators (shl and shr) so i could add blshift_i and brshift_i templates. the blshift_i template is ok, but brshift_i causes a segv in the nqp build pretty early on 09:15
gist.github.com/MasterDuke17/6529d...363b16b392 if anyone is interested 09:19
moon-child MasterDuke: you want sar there, not shr, I think 09:24
(I doubt that's the problem, but) 09:25
sar is signed shift, shr is unsigned
MasterDuke gist.github.com/MasterDuke17/6529d...-patch-L75 i do actually use sar, but i named the operator shr just for consistency with shl (open to changing the name) 09:26
moon-child oh, you do have sar. Nevermind
yeah
MasterDuke what's weird is that the segv seems to be in compiling the tiles, not because it should have been shr instead of sar and the nqp build saw some incorrect values 09:30
nine So what is it actually segfaulting on? 09:45
MasterDuke gist.github.com/MasterDuke17/6529d...og-L24-L27 09:46
jnthnwrthngtn moarning o/ 09:50
Nicholas \o 09:55
nine MasterDuke: what does the reg 3 mean? 09:58
MasterDuke i think 3 is the "cost" 09:59
github.com/MoarVM/MoarVM/blob/mast...-templates 10:00
timo moarning :)
MasterDuke oh, i also had sh(l|r)_reg, but those cause some "bad operand mode in `shl rq,rq'" dynasm error when building moarvm 10:07
do sh(l|r) not take quads?
timo hm. i don't think it makes terribly much sense to shift by more than the width of your widest register? 10:09
Nicholas IIRC it's undefined behaviour in C, at least for signed integer types 10:10
to shift wider than the variable
(and if you try it, you'll find that some CPUs truncate the shift value to the size of the register, so a 32 bit shift on a 32 bit CPU is a shift of 0, so no-op, whereas others truncate it to 8 bits, so a shift of 32 results in a register containing 0, whereas a shift of 256 is a no-op) 10:11
(but as it's undefined behaviour, the optimiser will exploit it to not care, and wierder shit will actually happen. It's about as fun as a de-opt bug) 10:12
MasterDuke maybe i need something like github.com/MoarVM/MoarVM/blob/mast...sc#L64-L79
nine If the compiler segfaults, it cannot be due to semnatics issues in your implementation. For me it was usually some mistake when writing templates like nesting errors, missing size definitions or the like 10:13
10:17 Kaiepi joined
MasterDuke they're pretty simple templates, e.g., `(template: blshift_i (shl $1 $2))` 10:25
nine The templates yes. And the JIT has become a bit more accomodating to such errors and gives proper error messages. I'd guess that for tiles it's still far less forgiving 10:26
MasterDuke i just tried replacing the 'Rq's with Rb, but no change 10:32
Geth MoarVM/new-disp: 4 commits pushed by (Jonathan Worthington)++ 11:41
12:03 reportable6 left
Geth MoarVM/new-disp: 3472b82064 | (Jonathan Worthington)++ | 6 files
3472b82064 | (Jonathan Worthington)++ | 6 files

Which involves mostly the same logic as those that invoke bytecode, at least for now. In the future we can look at doing better at this for some functions that can just become an op, and when we JIT them we may also find a way to avoid the MoarVM calling conventions entirely and just spit out C function calls.		 12:13
nine core.c stage parse is now at 53.514s. Used to be >70s here 12:18
jnthnwrthngtn Seems speshing helps. :) 12:19
timo we don't try to inline runbytecode yet, right? 12:20
jnthnwrthngtn timo: No, and when I added sp_runbytecode I forgot that I wanted an extra 16-bit int slot for spesh candidate too
timo mhm 12:21
jnthnwrthngtn We'll poke -1 into it when we spit out runbytecode
So the only thing the optimizer has to do is replace the -1 with the candidate index
To turn it into a pre-determiend specialization call
And once *that* works, yes, inlining next 12:22
Nicholas where does lunch fit into all of this?
timo i imagine that'll unlock a whole load of optimization, some of which we didn't have before
Geth MoarVM/new-disp: 25a8a8ab3c | (Jonathan Worthington)++ | src/spesh/disp.c
Translate MVMDispOpcodeSet
jnthnwrthngtn Nicholas: Already eaten
Nicholas you're doing better than ilmari. He forgot yesterday. 12:23
jnthnwrthngtn Oops
I have a very light breakfast, so tend to notice that I need lunch :) 12:24
jnthnwrthngtn hunts for the commit where timo did a draft of the attribute loading 12:25
nine Seems like I misremembered. Got 58.641s before the "Turn spesh back on" commit 12:26
12:26 brrt joined
nine those 70s were probably debug builds 12:27
MasterDuke i ran out of the really good granola i like yesterday, so had to have a bowl of special k. i have a tiny bowl of the granola and it keeps me full until lunch, but after the larger bowl of special k i was hungry a good bit earlier
but i did make a nice grilled ham+cheddar+spicy mustard, so all was not lost 12:29
timo jnthnwrthngtn: i don't think i did attribute loading, at least not this time around. perhaps last year, though? 12:30
brrt MasterDuke: did you make any progress with the shift-operators? 12:31
MasterDuke not since my last comment. have you looked at my gist?
jnthnwrthngtn timo: Yes, last year. Quite a lot changed :) 12:33
brrt I did look at the patch, I thought it looked reasonable enough, but I may be missing something 12:34
MasterDuke did the backtrace tell you anything?
i just rebuilt with --optimize=0 and it aborted at `assert(state_info != NULL);` 12:35
nchild was 2
lstate was 99, rstate was 36 12:36
nine comes back to IRC, bowl of granola in hand, reads about granola
MasterDuke brrt: does it matter where in src/jit/expr_ops.h i put the new ops? 12:40
brrt MasterDuke: that's probably because you have a tree that is (shift_left $a $b) where $b is not a (const ...) 12:42
MasterDuke but then how come only having mul_const is ok? 12:43
brrt ehm, I'd think that because there's also a mul with non-const, or we only ever generate mul-const trees 12:44
I'm sorry, I'm talking without context, and I should really look into it a bit more
MasterDuke i initially tried defining *_reg versions, but those gave that "bad operand mode in `shl rq,rq'" error from dynasm during moarvm compilation (same when i tried switching to Rd) 12:45
brrt hmm, let's see what gcc does :-) 12:52
MasterDuke: gcc gives me 'shl rax,cl' for 'x << y' 12:57
so the answer is 'the second operand has to be byte-sized'
MasterDuke matches github.com/MoarVM/MoarVM/blob/mast...1294-L1302
brrt hehe 12:58
unfortunately the tiler doesn't have a 'bail', it just crashes
this should really be fixed
I think it's on one of my TOOD documents from years back
MasterDuke `| shl Rq(reg[0]), Rb(reg[1]);` gives `error: bad operand mode in `shl rq,rb'` 12:59
get same with rq,rq and rb,rb
brrt which part gives that? dynasm? 13:03
dynasm can be wrong
13:05 reportable6 joined
brrt do you have a branch published? I could take a look :-) 13:05
MasterDuke i'll push it now 13:11
brrt ty
no promises though :-)
oh, I found your issue 13:12
c9x.me/x86/html/file_module_x86_id_285.html
operand register has to be 'cl'
I ... forgot what, if any, plan I had to spec such register requirements. 13:13
MasterDuke github.com/MasterDuke17/MoarVM/tre..._templates
brrt because ideally the register allocator can handle that
it may have been that register requirements were the blocker for these templates
MasterDuke oh, huh 13:14
brrt you can work around this in the tile, by conditionally storing/restoring the value of rcx if it isn't free, I think there are some examples of that 13:15
but it's not ideal
MasterDuke rcx is cl? 13:22
brrt yes, cl is the lower 8 bits of rcx/ecx/cx 13:23
I'm somewhat surprised at how much I enjoy this kind of esoterics 13:24
MasterDuke heh. i enjoy reading about assembly programming more than i enjoy doing it 13:25
are there any registers it will always be safe to store rcx into? 13:29
brrt well.. let me look up what mechanism I used 13:47
MasterDuke gist.github.com/MasterDuke17/6529d...363b16b392 updated with a non-working patch 13:57
gist.github.com/MasterDuke17/6529d...363b16b392 updated with a non-working patch 13:58
brrt not sure why you do the `test` bits 13:59
I mean, it could be a legitimate zero in there 14:00
MasterDuke hm, right
removed the tests and jumps, just unconditionally move and move back. but not just get `bad operand mode in `mov i?,rb':` for `mov CL, Rb(reg[1]);` 14:02
14:02 brrt left
MasterDuke *now 14:02
14:03 brrt joined
brrt sorry, was disconnected 14:03
I think you have to use 'cl', lowercase, otherwise dynasm thinks you're using not a register name 14:04
further, the non-allocatable register is 'rax'
so rather than 'r11' you do a `mov rax, rcx' before and `mov rcx, rax` after
Geth MoarVM/new-disp: 4 commits pushed by (Jonathan Worthington)++ 14:05
MasterDuke ooo, now moarvm builds... 14:06
but the nqp build hangs...
brrt :-( 14:07
but, progress
MasterDuke gist updated with a backtrace 14:09
and what i have for the tile.dasc now 14:11
brrt hmm 14:16
I'll have to look at it some other time I'm afraid
MasterDuke now worries, thanks for the help so far
*no worries
brrt happy to :-) 14:17
Geth MoarVM/new-disp: 1e58c8a7ed | (Jonathan Worthington)++ | src/spesh/disp.c
Translate remaining value result ops
 14:25
Nicholas ooooh, tanks *and* cranes. We are so blessed today: www.youtube.com/watch?v=sTA0GTgFn5E 14:39
brrt what are they launching today? 14:41
MasterDuke the "NO LAUNCH TODAY" text makes me suspect we're just watching construction? 14:43
Nicholas nothing (intentionally). It's "Ground Support Equipment" moving (a fuel or oxygen tank) 14:44
and the big crane is building the launch tower
brrt ooh :-) 15:02
and this is for the SpaceShip mega-rocket?
Geth MoarVM/new-disp: 7ce43d7e06 | (Jonathan Worthington)++ | 8 files
Translate object and string literal guards

Much of this thanks to timo++.		 15:04
jnthnwrthngtn That seems to get us translating all of the dispatch programs from monomorphic sites in the NQP build
Nicholas is your beer fridge ready to do this work justice? 15:06
timo \o/ 15:11
jnthnwrthngtn Yes :) 15:12
But it's still 2 hours or so until home time :)
dogbert17 Perhaps some tea in the meantime 15:14
Geth MoarVM/new-disp: df79c45cb7 | (Jonathan Worthington)++ | 4 files
Prepare sp_runbytecode* for spesh linking

Add a slot where we can include a pre-selected spesh candidate. In the event one is selected, we'll replace the -1 with the index of the selected candidate.
jnthnwrthngtn The tea I'm drinking is apparently past its use-by date. I didn't know that dried tea leaves would really have one... 15:16
nine Surprising things have them. Like table salt 15:17
jnthnwrthngtn That one I'm quite sure is silly. 15:18
15:18 evalable6 left, linkable6 left
jnthnwrthngtn I don't actaully know about the tea, but it seems...odd... 15:18
It tastes alright. I'm still alive after the first cup.
15:19 evalable6 joined
nine Well...actually :) Yes the salt is obviously stay the same as it's been for the million of years under ground before we dug it up. I specifically said table salt because that's usually iodised. And the iodine reacts with oxygen over time and loses its effect 15:21
MasterDuke jnthnwrthngtn: that spesh_cand is the index right? will this be a problem merging with the remove-cands-if-too-many-deopts pr, where instead of passing indexes around it was converted to passing the actual candidate 15:23
github.com/MoarVM/MoarVM/pull/1426...664b0db430 15:24
brrt iodine reacts with oxygen? as in NaI? 15:25
jnthnwrthngtn MasterDuke: Ugh, yes, it would :/
brrt and even if it did, so what? we can't absorb it's oxidized variant? I'm confused
nine Please don't ask such pointed questions :D It's not pure iodine that's in salt, it's some other chemical compound 15:26
brrt heh :-D I'd thought the other chemical compound was NaI, and I can be easily mistaken
nine I don't really remember the specifics. Just that I wondered about that date some years ago and dug into the question
jnthnwrthngtn MasterDuke: I don't think it'll be hard to adjust it later 15:27
brrt then you probably dug further than I ever have
MasterDuke jnthnwrthngtn: cool
16:00 vrurg left 16:04 MasterDuke left 16:07 vrurg joined 16:15 MasterDuke joined
Geth MoarVM/new-disp: e84398286c | (Jonathan Worthington)++ | 9 files
Reinstate spesh logging of target static frames

To achieve this, we also re-organize where spesh logging of dispatch program results occurs. This not only means we have the information in the right place, but also fixes us logging information about the entry to a new frame before logging the dispatch result that led to that frame entry, which could confuse the spesh stats handling.		 16:54
MoarVM/new-disp: 9e453e3fb9 | (Jonathan Worthington)++ | src/spesh/disp.c
Set literal value facts on loaded spesh slots
MoarVM/new-disp: 5290495f72 | (Jonathan Worthington)++ | src/spesh/disp.c
Place deopt all and cached anns onto runbytecode

The deopt all for correctness, and the cached one for being able to look up the target static frame in the spesh plan.		 17:13
nine Down to 47s! 17:15
This spesh things...its useful! 17:16
jnthnwrthngtn Before it was...54s?
nine 58 before the "Turn spesh back on commit" 53 early afternoon today 17:17
17:19 linkable6 joined
Geth MoarVM/new-disp: 988c345233 | (Jonathan Worthington)++ | src/spesh/optimize.c
Clean up after previous invocation opts a bit
 17:29
MoarVM/new-disp: 51372a035b | (Jonathan Worthington)++ | src/spesh/optimize.c
Start filling out runbytecode optimization

It doesn't actually optimize anything yet, however does do enough to get the target static frame that will be invoked, or that we assume will be, which is the first step towards spesh linking and then inlining.
jnthnwrthngtn That last commit mostly because it's hometime rather than becuase it's doing something useful yet. :)
bbl o/ 17:35
17:58 patrickb joined 18:02 reportable6 left 18:03 reportable6 joined 18:05 patrickb left
timo lmao "runcfunc" 19:21
i see at least one case where the first guard a disp program spit out was known to be unnecessary 19:23
jnthnwrthngtn I've seen a few cases of that too 20:22
I'm not sure why they aren't optimized away
timo my assumption is: ins pointer would need to be turned back one spot after optimizing the dispatch op 20:28
Did You Know™ that with dladdr or dladdr1 you can get a symbol's info given an address
jnthnwrthngtn Hm, I thought it was
timo maybe that can be used in the sslot print-out for BOOTCCode
jnthnwrthngtn timo: No, I didn't know that :) 20:29
timo we already use some dynamic library loading stuff for extops, and for nativecall; i'll want to see if whatever we're using exposes that in a crossplatform way somehow 20:30
hum. the first symbol i try it on: gdb knows its name, dladdr gives me the path to libmoar, but name and symbol address are set to null, aka "no symbol found here" 20:37
which i guess that function isn't a thing we export and that's what makes that happen?
with MVM_repr_elems it works just fine
so gdb must be going through the dwarf info which has info for pretty much all symbols, but dladdr will only have access to things that are exported :( 20:38
[Coke] src/core/callsite.c:127:75: warning: incompatible pointer types passing 'MVMString *' (aka 'struct MVMString *') to parameter of type 'MVMCollectable *'
(aka 'struct MVMCollectable *') [-Wincompatible-pointer-types]
MVM_profile_heap_add_collectable_rel_const_cstr(tc, snapshot, cs->arg_names[i],
timo yeah, just (MVMCollectable *) that and it'll be fixed 20:39
hey at least dladdr reliably gives me the base address where the elf file is loaded in memory, which means i can throw libdwarf at it :D 20:40
but ... ehh :\ 20:42
Geth MoarVM/new-disp: b773dfd7cf | (Jonathan Worthington)++ | src/core/callsite.c
Add a missing cast

Spotted by Coke++		 20:50
[Coke] I HALPED 20:56
Nicholas \o/
timo halpify 21:10
22:08 brrt left
Geth MoarVM/new-disp: 5eacf3e8ab | (Jonathan Worthington)++ | 9 files
Remove sp_resolvecode op

Which is no longer required with new-disp.		 22:14
dogbert17 someone is working late it seems 22:16
22:51 squashable6 left 22:52 squashable6 joined
Geth MoarVM/new-disp: 70106bc08f | (Jonathan Worthington)++ | src/spesh/disp.c
Add missing `static`
 23:13
MoarVM/new-disp: a6f55eec21 | (Jonathan Worthington)++ | src/spesh/optimize.c
Emit guardsf when speculating on static frame

Another step in porting the spesh pre-determination optimization to work on the runbytecode op. Since we already have a VM code ref, we don't have to deal with the logic around finding it; that just goes away from the optimizer.
jnthnwrthngtn Yeah, this bit is largely straightforward porting existing opts to the new-disp way of doing things
Geth MoarVM/new-disp: a48e1e5f0b | (Jonathan Worthington)++ | src/spesh/optimize.c
Locate spesh candidate for linking

This is enough that we successfully find a spesh candidate that we could link or inline in many cases. We don't yet do the linking, however, as the arg guard insertion still needs porting.		 23:28
jnthnwrthngtn OK, the arg guard insertion can be tomorrow, then hopefully spesh linking works. 23:31
'night o/ 23:32

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!