Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:05 reportable6 joined 00:12 squashable6 left 01:29 dogbert11 left 01:48 nebuchadnezzar left 02:06 frost joined 02:38 dogbert11 joined 02:41 dogbert17 joined 02:43 dogbert11 left 02:51 dogbert17 left 02:53 dogbert17 joined 03:06 nine left, nine joined 03:14 squashable6 joined 03:55 dogbert17 left 04:43 dogbert17 joined 05:08 dogbert17 left 05:47 dogbert17 joined 06:03 reportable6 left 06:05 reportable6 joined 07:10 dogbert17 left 07:12 nebuchadnezzar joined
nine Oh darn. Seems like I was so close when looking for a GC issue with resumptions. But that case MVM_DISP_OUTCOME_NEXT_RESUMPTION was handled MVM_disp_program_mark_outcome threw me off and I figured I just didn't understand it well enough 07:26
07:39 dogbert17 joined 08:02 dogbert17 left 08:14 squashable6 left 08:40 dogbert17 joined
Nicholas good *, #moarvm 09:02
jnthnwrthngtn: ASAN doesn't find any problems with that branch :-( 09:03
09:12 sena_kun joined 09:19 dogbert17 left 09:20 dogbert17 joined 10:33 evalable6 left, linkable6 left 10:35 evalable6 joined, linkable6 joined
jnthnwrthngtn moarning o/ 10:39
lizmat jnthnwrthngtn o/
this just on #raku-dev: *being annoying* Can someone comment on my last post on github.com/Raku/problem-solving/issues/186 so I can maybe took another shot at it tomorrow (I am trying to do like Zoffix a while ago with Rotfix Thuesday) x) 10:40
thought I'd repeat that here just in case
jnthnwrthngtn
.oO( Thuesday would be a good name for Wednesday... )
 10:43
Geth MoarVM/new-special-return-only: 3d213634e9 | (Jonathan Worthington)++ | src/6model/parametric.c
Migrate parametric type usage of special return
 10:49
jnthnwrthngtn grmbl, I don't seem to be able to repro the CI failures of new-special-return on my office box either 10:52
lizmat OOC: will nqp::p6scalarwithvalue continue to be a thing ? 11:25
jnthnwrthngtn I'm not aware of it causing any bother, so probably 11:26
11:26 Altai-man joined
lizmat ok, cause JSON::Fast uses that 11:27
Geth MoarVM/new-special-return-only: 099e18527c | (Jonathan Worthington)++ | src/core/args.c
Migrate bind failure usage of special return
MoarVM/new-special-return-only: cdb6374435 | (Jonathan Worthington)++ | src/debug/debugserver.c
Migrate debug server usage of special return
jnthnwrthngtn lizmat: meh, why
lizmat to makes it faster ? 11:28
jnthnwrthngtn Faster than what?
lizmat than before ?
jnthnwrthngtn m: for ^1_000_000 { nqp::p6scalarwithvalue(42) }; say now - INIT now 11:29
camelia 5===SORRY!5=== Error while compiling <tmp>
Could not find nqp::p6scalarwithvalue, did you forget 'use nqp;' ?
at <tmp>:1
------> 3 ^1_000_000 { nqp::p6scalarwithvalue(42)7ā5 }; say now - INIT now
jnthnwrthngtn m: use nqp; for ^1_000_000 { nqp::p6scalarwithvalue(42) }; say now - INIT now
camelia ===SORRY!===
Unknown QAST node type NQPMu
jnthnwrthngtn sigh, how is it even used
Oh, it wants a descriptor...how does JSON::Fast get one of those?
lizmat first param is the descriptor
from the Hash or Array it has created 11:30
jnthnwrthngtn Ah, `my $descriptor := nqp::getattr(%result,Hash,'$!descriptor');` 11:31
Is this really a lot faster than going via ASSIGN-POS... 11:32
lizmat fwiw, I was looking at how easy it would be to have JSON::Fast create an immutable data structure
rather than hashes and arrays, but Maps and Lists without containers
jnthnwrthngtn Yeah, that'd not be a drop-in replacement for JSON::Tiny, which is in JSON::Fast's job description 11:33
But certainly as an alternative it'd be worth it
lizmat with a parameter of course
from-json $text, :immutable
jnthnwrthngtn I'd also pondered a JSON::View or some such that lazily parses 11:35
Or at least lazily ASTs :)
So you'd not get arrays/hashes/maps/whatever, but rather an object graph that does Associative and Positional in the appropriate places and pulls the bits out on demand 11:36
lizmat yeah, gotcha, but that'd be more involved atm and more than I would like to sign up for atm :-)
jnthnwrthngtn Yeah, it'd be fun, but my copious free time has a lot of competition :) 11:37
nine Copious free time? How does one get that? 11:39
jnthnwrthngtn I thought the saying had the implication of "lack of" automatically :D 11:41
nine I intentionally ignored the implied bit to take it a step further :D 11:44
11:46 squashable6 joined
nine After digging out your issue about generate-and-export pattern github.com/Raku/problem-solving/issues/159 I got that "so much interesting things, so little time" feeling as well 11:47
jnthnwrthngtn :) 11:56
Looks like the first 5 commits of the special return thing are going to turn out to be innocent
nine So, progress at least? 11:59
jnthnwrthngtn Well, they ain't much use of their own, but I guess :)
12:02 reportable6 left
Geth MoarVM/new-special-return-only: ae6b4b7414 | (Jonathan Worthington)++ | src/core/frame.c
Migrate return handler usage of special return
 12:04
jnthnwrthngtn Now for this one.
12:05 reportable6 joined
jnthnwrthngtn nine: How close is the new-disp based native call to being ready? 12:21
nine I've gotten distracted a little from it. I think all it needs is a bit of cleanup and proper commit messages 12:22
jnthnwrthngtn releasable6: status
releasable6 jnthnwrthngtn, Next release in ā‰ˆ16 days and ā‰ˆ6 hours. 3 blockers. Changelog for this release was not started yet
jnthnwrthngtn, Details: gist.github.com/28a3fc65e0907dd455...f980f8ae17
12:38 CaCode joined
jnthnwrthngtn The return handler woulda been a good candidate for causing the breakage...but it's not the one 12:46
Geth MoarVM/new-special-return-only: 94778d2bd2 | (Jonathan Worthington)++ | 4 files
Migrate exception unwind usage of special return
nine jnthnwrthngtn: would have been a good candidate. But OTOH what could have been wrong about that? It's small and there's nothing special about it. 12:48
So maybe it's goot that it wasn't that commit
jnthnwrthngtn I guess, yes. :) 12:50
nine Now the unwind commit looks scary. A wrong tc->interp_reg_base e.g. could cause bizarre errors 12:51
jnthnwrthngtn Yeah, this one would hurt to debug for sure. So far, 11 successful, 7 pending 13:04
15/3...looking less likely 13:07
Nicholas nine: currently nqp 0dc60582b7d01c8dd6e357cb25167f0232be9157 (origin/new-disp-nativecall) doesn't build on MoarVM e2f7edce746275714fea5f0b8402c35c3d465ec3 (origin/new-disp-nativecall) 13:14
jnthnwrthngtn 17/1... Of course it's the Windows one taking the time to finish 13:19
nine Nicholas: oh, forgot to push a commit. Thanks for the heads-up! 13:23
Geth MoarVM/new-special-return-only: 22e1926e10 | (Jonathan Worthington)++ | 2 files
Avoid duplicate interp sync-up on return

We only needed the case in remove_one_frame now because of lazy deopt, so just move it there instead, now that unwind frame is doing the whole thing.		 13:25
jnthnwrthngtn Next candidate :) 13:26
nine That could be a classic "oh there's this one other case that I didn't think of" error 13:28
jnthnwrthngtn Already one failure, but it's a git fetch one 13:32
13:38 frost left 14:29 CaCode left
jnthnwrthngtn Another one ruled out 14:31
And now it gets tricky because the next one is the finalizers one but there are a few fixes to that that came later... 14:33
Geth MoarVM/new-special-return-only: 3b1d3cc633 | (Jonathan Worthington)++ | 3 files
Don't use special return for finalizer running

For now, we'll handle this by checking if there's anything in the finalizer queue each time we exit a frame. It's not ideal, in that it costs something every frame exit, but it unblocks eliminating the special return mechanism, which will in turn unlock removing many other checks on frame exit, so it's a reasonable trade-off.		 14:35
MoarVM/new-special-return-only: a3a2dd4dff | (Jonathan Worthington)++ | src/gc/finalize.c
Avoid a NULL dereference when running finalizers

And also correct a comment.
MoarVM/new-special-return-only: 7917743eb6 | (Jonathan Worthington)++ | src/gc/finalize.c
Don't lose exception handler results

Invoking finalizers could end up replacing tc->last_handler_result, leading to missing or incorrect values being observed for that after finalizers had run. Don't run finalizers when there is such a risk.
Nicholas nine: with rakudo 14f4b722ec480925d39b584ceee3371e6dd2b4a9 I get: 14:39
$ ./rakudo-m -Ilib t/04-nativecall/22-method.t
1..2
===SORRY!===
Unhandled arg type 4
# You planned 2 tests, but ran 0
on x86_64, but ppc64 was fine 14:40
quite a few nativecall tests failed for me on x86_64, whereas ppc64 passed everything and ran the spectsts 14:41
have also seen:
===SORRY!===
Unhandled arg type 8
jnthnwrthngtn Seeems we have a winner. It's the fianlizer stuff after all. 14:46
nine Nicholas: sorry again. Turns out there was something in my git stash that needed committing and pushing to rakudo 14:47
Geth MoarVM/new-disp-nativecall: 30 commits pushed by (Stefan Seifert)++
review: github.com/MoarVM/MoarVM/compare/e...777f42c379
nine Just a rebase on current master ^^^
nine is now building everything again and checking that the published branches actually work 14:48
Nicholas I have blown away my installed tree, as I think I might have goofed on ppc64 14:49
nine Nicholas: what's published _now_ passes all tests for me 14:53
jnthnwrthngtn 7 failing. Yes, it is this. 14:59
But...I disabled running finalizers and still got such failures, I thought.
lizmat github.com/timo/json_fast/pull/78 # Add :immutable parameter to from-json 15:03
timo ^^ :-)
Nicholas $ ./rakudo-m -Ilib t/04-nativecall/00-misc.t 15:16
MoarVM oops in spesh thread: Malformed DU chain: writer sp_runnativecall_i of 10(0) in BB 5 is incorrect
DU backtraces from all of t/04-nativecall/00-misc.t t/04-nativecall/01-argless.t t/04-nativecall/22-method.t t/04-nativecall/04-pointers.t t/04-nativecall/05-arrays.t 15:20
Geth MoarVM/new-special-return-only: 60ab3bb064 | (Jonathan Worthington)++ | src/core/callstack.c
Try disabling finalization entirely
 15:21
Nicholas nine: that was before NQP did 791163b28...8b609200d 15:25
15:40 evalable6 left, linkable6 left 15:41 linkable6 joined
nine Fixed one DU chain issue (when running with nodelay) but now get one in allomorphs.pm6. Probably that one's there on master as well when running MVM_SPESH_BLOCKING=1 MVM_SPESH_NODELAY=1 ./rakudo-m -Ilib t/04-nativecall/01-argless.t with MVM_SPESH_CHECK_DU 1 15:50
MoarVM oops in spesh thread: Malformed DU chain: reader PHI of 5(8) in BB 20 missing 16:21
Now this one's interesting. Most of all because r5(8) is only read by 3 PHI nodes, but used nowhere else. And that's in the Before dump!
16:22 patrickb joined
japhb lizmat: Any interest in doing a similar PR for github.com/japhb/CBOR-Simple that you just did for JSON::Fast? (I'm head down in github.com/japhb/Terminal-LineEditor rebuilding part of it on top of github.com/japhb/Terminal-ANSIParser, so I'm trying to avoid being nerd sniped, though that definitely came close! :-) ) 16:34
lizmat "This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below." you evil person, you :-) 16:36
japhb lizmat: Um ... that seems not right 16:37
Which repo did you see that on?
lizmat github.com/japhb/CBOR-Simple/blob/...le.rakumod 16:38
japhb Oh heck, I bet I know what happened there. I probably cut-and-pasted an icon from my unicode explorer (it brackets characters with bidi lockdown so that RTL scripts won't mash the display). 16:39
Thanks for pointing that out. Now to chase down where it occurred. (And sorry about that.)
lizmat no worries 16:40
fwiw, any particular areas where you think optimization would be needed / most useful ?
16:43 evalable6 joined
japhb Ah-ha! Found you, you bastard! (It was in the docs, a unicode superscript that as I guessed was a copy-paste.) 16:44
Nicholas This is github reacting to the "new" CVE-with-a-logo-and-a-website? 16:45
i.redd.it/kgqdnb7poig31.jpg
lizmat :-) 16:46
japhb lizmat: I'm about to upload a fixed version. As for your question, I was thinking in particular of having an :immutable option to cbor-decode(). That said, if you find other optimizations, I'd love to hear of them. I usually use github.com/japhb/serializer-perf for perf testing, but it is *far* from exhaustive. 16:49
jnthnwrthngtn ooh, a segfault *locally* when I set MVM_GC_DEBUG=3 on the new-special-return branch 16:53
japhb CBOR::Simple pushed. I find it interesting that GitHub does not show that warning on the README, even though the BiDi control characters were copied there by mi6 ...
jnthnwrthngtn: Progress! Right? :-)
dogbert17 jnthnwrthngtn: could it be the one you fixed on master yesterday? 16:55
also is it the new-special-return branch or new-special-return-only branch which should be tested ? 16:58
17:00 patrickb left
jnthnwrthngtn dogbert17: new-special-return-only but without the final commit 17:06
nine jnthnwrthngtn: is seeing a register version only ever accessed by a PHI read something to worry about at all? Or is it just a side effect of how ssa transformation works? 17:12
jnthnwrthngtn nine: Sounds pretty normal to me; anything beyond the PHI would read the merged version produced by the PHI 17:14
Sigh, I suspect the SEGV I found is 1) legit, 2) easily fixed, 3) not the problem seen on the CI at all 17:26
Geth MoarVM/new-disp-nativecall: d6747783ca | (Stefan Seifert)++ | src/spesh/disp.c
Fix DU chain errors caused by optimization of native dispatch
 17:43
nine Nicholas: cannot provoke any DU chain issues anymore with this ^^^ 17:44
jnthnwrthngtn Pushed a fix to Rakudo for that SEGV
Really can't see it being the one to blame for the `make test` failures though 17:45
Nicholas nine: will test. However, can also say that new-disp-nativecall fails to link moar if Configure.pl is run with --has-libffi 17:46
in theory I could try to fix that, but was rebuilding your changes first
nine Oops...right, libffi is a thing. Need to port MVM_nativecall_dispatch to that 17:47
18:02 reportable6 left
Geth MoarVM/new-special-return-only: 658ef6f0a3 | (Jonathan Worthington)++ | src/gc/finalize.c
Try saving return info over finalizer runs
 18:07
jnthnwrthngtn Turns out that while I did fix a missing MVMROOT in the extops, it was unrelated to the thing that shows up on CI 18:09
I'm really confused; I can't repro it on two different machines, with GCC or with clang, with all kinds of GC nursery sizes. 18:10
But on the CI it happens pretty regularly
[Coke] maybe something tied to OS version or tooling in CI? 18:18
(can we find out what "hardware" it's running on?)
jnthnwrthngtn I dunno. I'm pretty fed up. 18:21
Wonder what other strategy I can find for running finalization
nine All the failures of the most recent run are "Type check failed for return value; expected CompUnit:D but got Method (method sink (Mu: *%_...)" 18:29
Could it be that finalizers overwrite the return value of the frame we're returning from? 18:30
jnthnwrthngtn That's what I thought, but a) why'd that not happen with equal frequency on my machine, b) we explicitly clear ->return_value and set ->return_type to void, so that the invoked code won't do that 18:31
nine Where do we clear those? 18:32
jnthnwrthngtn MVM_frame_dispatch_from_c(tc, handler, args_record, NULL, MVM_RETURN_VOID);
The second two args
Geth MoarVM/new-special-return-only: 6fcedf7b11 | (Jonathan Worthington)++ | src/core/callstack.c
Try not running finalizer if there was a thunk
 18:41
jnthnwrthngtn home time & 18:46
18:48 Altai-man left 19:03 reportable6 joined
jnthnwrthngtn Oh. That one passes! 19:29
timo ... a thunk in the night ... 19:33
lizmat pings timo with github.com/timo/json_fast/pull/78 # Add :immutable parameter to from-json 19:35
Geth MoarVM/new-special-return-only: c1971ad86c | (Jonathan Worthington)++ | 5 files
Remove legacy special return mechanism

In turn making the new one temporarily public, since a Rakudo extop relies upon it.
jnthnwrthngtn So this is going to get interesting when we get to the commit that eliminates the thunked flag... 19:36
But a couple I can pull in before that
20:28 evalable6 left, linkable6 left 20:31 linkable6 joined
Nicholas nine: it does fix the DU problems 20:34
Geth MoarVM/new-special-return-only: 92e829cefb | (Jonathan Worthington)++ | src/core/callstack.h
Add temporary define for migrating Rakudo extops
 20:38
jnthnwrthngtn Heh, massive fail without that :)
[Coke] lizmat++ 20:42
lizmat [Coke] huh?
thank you, but but but ....
[Coke] (for the pull request) 20:43
lizmat aaah... ok, dropped out of my head already :-) 20:46
japhb EBRAINTOOFULL 20:54
Geth MoarVM/new-special-return-only: b6523deb5d | (Jonathan Worthington)++ | 2 files
Eliminate redundant checks/logic during return

  Thanks to having a callstack record type to indicate the boundary of a
nested runloop (used in NativeCall callbacks), we no longer need to explicitly check for the threaded interpreter boundary anywhere. We can return NULL from the callstack unwind function, upon encountering the native call boundary, to indicate interpreter exit instead. This gets rid of a bunch of logic in remove_one_frame, and a little logic in exit_frame.		 21:14
21:30 evalable6 joined 21:34 sena_kun left
jnthnwrthngtn With that one fail, but I've seen it on master too. 22:24
Geth MoarVM/new-special-return-only: 7d4d163d25 | (Jonathan Worthington)++ | 2 files
Move clearing of ->work into unwind

We can elide it in the case it is a stack frame, since we're going to be removing it anyway.		 22:25

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!