Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
timo there's nothing yet to dump disp program related data? 09:49
jnthnwrthngtn timo: There's two `#define`s in src/disp/program.c; one dumps the outcome of recording the dispatch, the second dumps the dispatch program that was produced and installed. 09:53
timo i wonder how i missed that
(i only looked for "dump", and only in a few files, maybe that was it)
jnthnwrthngtn I sorta wish they each added the line/file of the callsite 09:54
timo i wonder if i can get that in there without too much reading of the everything 09:57
so i'm just randomly browsing a spesh log of core setting compilation
and i'm seeing ACCEPTS from QRegex with the type tuple QRegex (Conc), BOOTStr (Conc), the code is basically just "check params are ok, call `parse` method on object" 09:58
and the parse method call is this here: [Annotation: Cached (bytecode offset 60)] 09:59
dispatch_o r6(3), lits(lang-meth-call), callsite(0x1721d20, 8 arg, 4 pos, nonflattening, interned), r4(2), r5(1), r3(2), r2(2), r6(2), r7(1)
but in the spesh log the stats don't point out any dispatcher outcomes at all, and the After version says "never dispatched"
*looks at impl of lang-meth-call* 10:00
jnthnwrthngtn Yes, if you look at iirc MVM_frame_dispatch then you'll see it never sets a spesh correlation ID or records params etc. 10:01
That may be LHF-ish
timo oh my! that's a TLA that i like
jnthnwrthngtn Also spesh candidate selection
The main thing to know is that the calling conventions are a bit different now 10:02
And the adaptation needed will be there. Given MVM_frame_invoke is going away once we fully switch over to MVM_frame_dispatch, and most things are switched over anyway, you could just comment out the spesh bits in _invoke rather than finding a way to make it work with both at once 10:03
Also if you're wondering how any of the dispatch works and didn't ready the 6guts posts, I think at least one of them is a decent introduction :) 10:04
timo github.com/MoarVM/MoarVM/blob/new-...2420-L2425 here the implementation of MVMDispOpcodeGuardNoResumptionTopmost falls through to MVMDispOpcodeGuardNoResumptionCaller falls through to MVMDispOpcodeUpdateResumeState, that smells unintentional, but if it is intentional, i'd put a comment for you 10:05
jnthnwrthngtn I think that's accidental 10:06
timo i'll cook up a commit quickly
jnthnwrthngtn Yeah, it is.
...wonder how we get away with it o.O 10:07
Please at least check for no change in NQP and Rakudo `make test`
Just in case
Next task for me is turning smart_intify (and later smart_numify) into dispatchers 10:08
timo that doesn't hang very low, does it?
i guess it depends mostly on when you're going to do the next bit of new-disp work
jnthnwrthngtn Oh, I was going to do this right now :) 10:09
It will be one more `make test` failure eliminated.
And maybe more spectests passing
I probably should work on something else at some point today, but this is rather good fun :) 10:10
Nicholas the night is still young 10:12
jnthnwrthngtn :)
timo t/nqp/097-hll.t ........................ 1/50 lang-meth-call cannot work out how to dispatch on type '(null)' 10:15
i presume no test failures of nqp tests are known
i get this without my local changes as well, though 10:18
Nicholas I see that one fail. I assumed that it was a known failure
timo there's a few nok followed by that exception 10:19
OK, then that's probably fine. at least jn surely knows about it now if it's not
jnthnwrthngtn That one is a known failure
There's a minor design decision that I'm procrastinating :)
Nicholas aha right, thanks. that answered my next question 10:20
timo ah ISTR something about that
jnthnwrthngtn (Also fixing it almost certainly doesn't win me anything in Rakudo tests)
timo ok rakudo's make test output is identical 10:28
jnthnwrthngtn yay
timo++
I see the libera move gave us opposite outcomes on nick length. 10:29
Must be some conservation principle at work.
Nicholas OK, it's still registered, but it's not used. 10:30
(I forgot to chcek with whowas before pulling that little stunt)
jnthnwrthngtn Yeah, guess maybe some day if it's unused long enough I might get it back
Nicholas so I wonder when the next round of expiry will be
they announced that they changed policy so that registered but unused nicks would expire rapidly. But I don't think that they made that policy retroactive. 10:31
Geth MoarVM/new-disp: 5e7743cdc3 | (Timo Paulssen)++ | src/disp/program.c
Add unintentionally omitted break to cases
MoarVM/new-disp: a284a6d5e2 | (Jonathan Worthington)++ | src/disp/syscall.c
Add can-unbox-to-* syscalls

This will save the various coercey dispatchers from doing a rather more expensive attempt to unbox followed by exception handling to see if it's possible.
10:35
timo the disp dump also doesn't output what the name of the dispatch target was that launched the recording? like froj must recording and program you can't tell that it was a boot-boolify? 10:43
jnthnwrthngtn True 10:44
timo i assume you'd also like that?
especially boot-boolify can be sneaky i guess 10:45
jnthnwrthngtn Yeah, it can be useful info 10:48
Geth MoarVM/new-disp: 560d22df24 | (Jonathan Worthington)++ | src/disp/syscall.c
Further syscalls to aid porting smart_intify
10:49
jnthnwrthngtn Yup, down to 4 files with failure in make test after porting smart-intify 10:57
I'll do smart-numify while I'm at it 10:58
timo mhh, the current_disp of the record will contain whatever we ended up calling, so would always be like boot-code or boot-constant etc, let's see where i need to look to find the first dispatcher in the chain instead 11:06
jnthnwrthngtn Yeah, 'fraid so 11:07
Lunch, bbiab
Geth MoarVM/new-disp: 3b5dac2315 | (Timo Paulssen)++ | src/disp/program.c
For dispatch recording and dispatch program dump, add filename and line number info
11:13
timo here's the debug output addition
sometimes looks unhelpful, for example at <unknown>:1 (<ephemeral file>:<mainline>) 11:16
but what can you do
how do we distinguish "unboxability" from "unboxability", i wonder. 11:25
as in, what can be unboxed, and what can't be boxed
i guess the latter would be inboxability 11:26
and the combination would be inunboxability
AFKabit 11:27
MasterDuke timo: is the profiler going to need any changes to support new-disp? 11:36
jnthnwrthngtn I suspect some of the missing line/file info is because we don't emit those annotations properly for rules 11:52
MasterDuke is that something can be (easily) fixed? 11:54
jnthnwrthngtn Yes 11:55
Geth MoarVM/new-disp: 998bbc01ff | (Jonathan Worthington)++ | src/disp/syscall.c
Further syscalls to aid porting smart_numify
12:01
jnthnwrthngtn Either we are emitting a Stmts node around the regex node but not setting a match on it (and I think fixing that might do it), or alternatively we need to emit annotations on compilation of some regex nodes 12:02
MasterDuke would this be a fix for master also? or just new-disp? 12:03
jnthnwrthngtn master 12:04
It's independent of new-disp 12:05
I'd meant to look into it before because it also blocks comma being able to do code coverage of grammar rules
MasterDuke i'm up for small, self-contained projects while on holiday 12:06
have an idea of where in the (i assume rakudo) source to start looking? 12:07
jnthnwrthngtn Starting from the regex_def action method, if it's just a failure to stick a node on the QAST::Stmts 12:08
Otherwise look into compile_annotation and where it's used 12:09
(In the NQP repo, under src/vm/moar/QAST/) 12:10
MasterDuke cool, thanks
timo, jnthnwrthngtn: and what's the sample code you used that exposed the missing annotations? 12:11
jnthnwrthngtn Down to 3 failures in `make test` and up to 1296 fully passing spectest files 12:19
MasterDuke: If you have a grammar and run with MVM_COVERAGE_LOG=foo or some such, then `foo` will miss entries for the rule, iirc 12:20
Ohh, it may not be regexes in general...not sure 12:23
But
./rakudo-m --ll-exception -e 'grammar G { token TOP { (<xxx>) } }; G.parse("x")'
Oh, it is in fact
In the output of that:
from <unknown>:1 (<ephemeral file>:)
from -e:1 (<ephemeral file>:TOP) 12:24
Ah, no, it knows there TOP is. But the (...) (which compiles into its own block) gets no annotation
dogbert17 (and others): I think new-disp is now at the point where it is becoming useful to look at remaining test (especially spectest) failures, and 1) report anything that is SEGV, 2) golf non-SEGV ones. For now, performance info is not useful (need to wire spesh back up). 12:39
I've not actually seen a SEGV for a while :) 12:40
.oO( cue report of 20 segfaults... )
12:41
haha...I just found one (heap profiler explodes) 12:46
ah, a silly one too 12:47
dogbert17 tries to build new-disp 12:48
jnthnwrthngtn Also, just fixed passing Proxy to `is rw` args in multiple dispatch
1306 fullying passing spectest files
I think that means less than 100 failing
dogbert17 noooo, I almost managed to build everything and then ... 12:59
+++ Generating perl6-valgrind-m
+++ Creating installation directories
+++ Removing old files
MoarVM panic: Cannot handle new callsite format in MVM_args_copy_callsite
make: *** [Makefile:1450: m-install-pre] Error 1
hmm, perhaps it just made it and then failed during some cleanup 13:00
jnthnwrthngtn ah, did you try to `make install`? 13:01
dogbert17 yes
jnthnwrthngtn ah yes, that is indeed still busted
`make test` and `make spectest` work without it though
dogbert17 cool, do I have to set any environment variables or is new disp used by default? 13:02
jnthnwrthngtn By default now 13:03
dogbert17 cool, three failed tests when running 'make test' but you mentioned that 13:05
jnthnwrthngtn Yes, matches what I have here 13:21
Hopefully will soon be down to 2 13:22
Geth MoarVM/new-disp: da6fd4a5a6 | (Jonathan Worthington)++ | 8 files
Fix callstack marking in heap snapshot mode

Which means that we no longer segfault when making a heap snapshot (or at least, not in the basic tested case).
13:24
jnthnwrthngtn Yup, that makes it 2 for me
MasterDuke why do only QAST::Stmt and QAST::Stmts nodes have a `self.compile_annotation($node);` in them? 13:32
jnthnwrthngtn Because statement level line numbers are considered accurate enough (and keep the size of the position info emitted down) 13:39
MasterDuke k 13:42
dogbert17 some tests complain about requiring mutable arguments even though they seem to have that? 13:45
Cannot resolve caller postfix:<-->(Int:D); the following candidates
match the type but require mutable arguments:
(Mu:D $a is rw)
(Int:D $a is rw --> Int:D)
jnthnwrthngtn Are you at ffede8cc0be? That fixed a bunch of those 13:46
(It's the latest commit I pushed)
dogbert17 aha, lemme check
dogbert17 I wasn't oops 13:47
jnthnwrthngtn I've just done a local fix for S02-types/num.t too (need to make sure it doesn't regress anything) 13:48
Also some regex tests blow up with "Unexpected named argument 'result-int' passed" and I've got a probable fix for that locally too 13:49
[Coke] wonders if it would help to get a blin run for new-dispatch before spectest hits 100% 13:52
jnthnwrthngtn Probably a bit early for that now, but yes, once we're within 10 tests or so, it could be an interesting indication 13:54
dogbert17 'new-disp callwith nyi' I guess that one is trycky 13:59
*tricky
Altai-man considers a good idea to do a run once spectest/stresstest are clean 14:00
jnthnwrthngtn dogbert17: yes, that's both tricky and on my todo list :) 14:02
dogbert17 ok, what about 'take without gather' then? 14:03
jnthnwrthngtn That's quite unexpected 14:05
Worth golfing 14:06
One thing worth noting: there may be spectests that are against fixes in master that we haven't got in new-disp yet (didn't rebase for a copule of weeks) 14:07
Altai-man: Might be worth having it slightly ahead of that if the handful of failures are relatively esoteric, just to get a picture of how much ecosystem fallout there is and start to categorize it into "we have to fix it", "the module depends on impl details but we can bridge it", "the module depneds on impl details and we can't do anything about it", etc. 14:09
Up to 1313 passing spectests for me 14:10
dogbert17 that's very close 14:12
[Coke] m: say 1313/1406 14:17
camelia 0.933855
dogbert11 here's a MoarVM Panic: squish(Any, 'a', 'b', 'b', 'c', 'd', 'e', 'f', 'f', 'a') 14:28
[Coke] is that on new-disp?
jnthnwrthngtn ah, that's the same one as with `make install` 14:29
dogbert11 yes, seems to be
[Coke]: yes it's on new-disp 14:30
dogbert11 this also fails with new-disp: multi f((Int :$value, *%)) { "Int $value" }; multi f((Str :$value, *%)) { "Str $value" }; f("a" => "x") 15:01
jnthnwrthngtn Ah, sub-signature unpacking. 15:02
Hm, maybe I have a guess about that 15:03
Geth MoarVM/new-disp: 2c0acef072 | (Jonathan Worthington)++ | src/core/args.c
Implement MVM_args_copy_callsite for new convs
15:18
jnthnwrthngtn With that, `make install` works 15:19
dogbert11 and you fixed at least one spectest file 15:21
jnthnwrthngtn ah, nice 15:23
jnthnwrthngtn Hm, where did the ~1405 number for spectests come from? Maybe it includes Inline::Perl5 tests? 15:28
for me it reports "Files=1349" at the end. I count 1321 `ok`s in the output. I see about 30 failed tests in the list of results.
m: say 1321 / 1349 15:30
camelia 0.979244
jnthnwrthngtn At least 2 failing files are due to needing to rebase
dogbert11 perhaps 1405 was a stresstest, hmm 15:31
jnthnwrthngtn Maybe 15:32
Anyway, seems to be it's looking rather better than we thought so far as spectest goes 15:33
I was assuming I had
~80 left to fix (without considering Inline::Perl5). There's actually more like ~30.
MasterDuke77 should a QAST::Regex ever not have a `:node($/)`? 15:35
jnthnwrthngtn It's optional; the current compiler frontend is quite enthusiastic about placing them, but the RakuAST-based one will be far less eager 15:36
MasterDuke77 hm
dogbert11 I have 35 test failures ATM 15:43
a bunch seems to be tests expecting a certain exception but gets AdHoc insted 15:44
e.g. t/spec/S32-str/sprintf.t 15:46
jnthnwrthngtn Hm, that seems to be the same as one of the remaining `make test` issues 15:49
dogbert11 here's a golf (I think): { sprintf "%d", 0^1 } 15:52
under new-disp I get: elems requires a concrete object (got a VMNull type object instead) 15:53
it is expected to return: Directive %d not applicable for value of type Junction (one(0, 1)) 15:54
nine 192 15:58
err...disregard that :) 15:59
Nicholas that's a bit short for a password
even our combination bike locks have more digits than that :-) 16:00
jnthnwrthngtn dogbert11: ah, thanks for the golf, that's easier 16:01
dogbert11: I've just done a debugging tweak in NQP to try and help figure out what on earth is going on
[Coke] jnthnwrthngtn: my number just a wc -l off the file minus comments, I was super lazy! 16:06
jnthnwrthngtn [Coke]: Ah, right, that includes tress tests then :) 16:10
dogbert11: Might have figured that one out just now
dogbert11 cool, a thinko? 16:13
jnthnwrthngtn Probably 16:14
I fixed something in the NQP location that threw the exception and misunderstood what it was meant to be sending along 16:15
dogbert11 so it might be an easy fix then? 16:16
jnthnwrthngtn Think I already have one
Well, it fixes your golf.
dogbert11 there are half a dozen files where tests fail due to incorrect exceptions being returned 16:17
jnthnwrthngtn And it fixes one more `make test`, and S32-str/sprintf.t
dogbert11 yay
jnthnwrthngtn Yeah, I suspect those all need different fixes, alas
dogbert11 darn
jnthnwrthngtn Fix for the sprintf thing pushed (to the nqp repo) 16:19
dogbert11 todays shocker is that I have a non alcoholic beer in my fridge
jnthnwrthngtn Tomorrow's shocker will be that I have lots of nice beer in the fridge but will have been told not to drink 16:20
dogbert11 with 30 degrees C (86F) both inside and outside I'll sonn have to open it 16:21
jnthnwrthngtn :)
Ouch, that's way too hot
dogbert11 yeah, it's not supposed to be this hot. Guess I shouldn't complain though :) 16:22
jnthnwrthngtn I fear that the things that are LHF-ish for me to fix are running out in the spectets now. Everything looks like something that needs non-trivial effort. :)
I fear that the things that are LHF-ish for me to fix are running out in the spectets now. Everything looks like something that needs non-trivial effort. :) 16:23
dogbert11 noo
jnthnwrthngtn Hmm, S05-modifier/counted-match.t is curious 16:24
dogbert11 is rebuilding nqp and rakudo 16:26
jnthnwrthngtn Hm, it vanishes upon golf. grr
ah, it golfs to "aa" ~~ m:nth(1|2)/a/; "aa" ~~ m:nth(1^2)/a/; 16:30
[Coke] has new beer that's a passionfruit mango randle. 16:33
errrrrr a shandy. oops 16:34
jnthnwrthngtn Hm, I wonder if the assignment is accidentally auto-threading... 16:39
oh or hmm
OK, now I'm at the "how did this ever work" point :) 16:40
m: "aa" ~~ m:nth(1|2)/a/; "aa" ~~ m:nth(1^2)/a/; 16:41
camelia ( no output )
jnthnwrthngtn m: dd $/; "aa" ~~ m:nth(1|2)/a/; dd $/; "aa" ~~ m:nth(1^2)/a/; dd $/;
camelia Nil $/ = Nil
Junction $/ = any(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2)))
Junction $/ = one(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2)))
jnthnwrthngtn We have this function: 16:45
method !match-nth(Mu \slash, \cursor, \move, \post, $nth, %opts) {
Signature binding fails because $nth is Junction, not Any, and so triggers junction auto-threading
And at least on new-disp, it sees that slash contains a Junction and so goes auto-threading over that, and then over $nth 16:46
I'm guessing it doesn't on `master`
m: dd $/; "aa" ~~ m:nth(1|2):i/a/; dd $/; "aa" ~~ m:nth(1^2):i/a/; dd $/; 16:55
camelia Nil $/ = Nil
Junction $/ = any(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2)))
Junction $/ = one(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2)))
jnthnwrthngtn m: dd $/; "aa" ~~ m:nth(1|2):x(1)/a/; dd $/; "aa" ~~ m:nth(1^2):x(1)/a/; dd $/; 16:58
camelia Nil $/ = Nil
List $/ = $(any(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2))),)
List $/ = $(one(Match.new(:orig("aa"), :from(0), :pos(1)), Match.new(:orig("aa"), :from(1), :pos(2))),)
jnthnwrthngtn is confused 17:00
Ah well, time to go home. 17:01
dogbert11 you've made a lot of progress today 17:03
jnthnwrthngtn This week has been pretty productive for new-disp 17:04
dogbert11 indeed
japhb jnthnwrthngtn++ # Heck yeah
jnthnwrthngtn Doubt I'll get to anything more tomorrow 17:07
home time &
Geth MoarVM: 97948c2a6f | (Ben Davies)++ | src/io/syncsocket.c
Assign the result of a socket(2) call immediately

This allows the socket to be closed should bind(2) or connect(2) fail.
18:16
MoarVM: d523b1fcf1 | (Patrick Böker)++ (committed using GitHub Web editor) | src/io/syncsocket.c
Merge pull request #1505 from Kaiepi/set-syncsocket

Assign the result of a socket(2) call immediately
dogbert11 jnthnwrthngtn: the 'take without gather' problem in new-disp I was harping about before can be 'golfed' down to: 18:17
MVM_SPESH_BLOCKING=1 MVM_SPESH_NODELAY=1 ./rakudo-m -Ilib -MTest -e 'is do { [+] grep * %% 2, (1, 2, *+* ...^ * > 4_000_000) }, 4613732, "fibonacci"'
patrickb Any reason not to give Kaiepi a commit bit in moar? 18:19
^ping jnthnwrthngtn
nine jnthnwrthngtn: LOL I fixed t/spec/S02-names/is_cached.t on new-disp :) Feel free to throw away my commit if you have something else in mind. It's more an exercise in getting the lay of the land anyway ;) 18:40
jnthnwrthngtn nine: Nice! Seems reasonable to me at first glance. 20:11
nine: One thing you may have an idea about is why ProxyReaderFactory's code gen ends up causing reproducible-builds.t to fail. 20:13
I don't think the things its producing should end up anywhere in the build output, but apparently somehow they do. 20:14
MasterDuke ugh, testing my changes is much more annoying on the laptop, building rakudo takes twice as long as on the desktop 20:20
jnthnwrthngtn: i added a `$block[1].node($/)` here github.com/rakudo/rakudo/blob/mast....nqp#L4962 when $block[1] was gist.github.com/MasterDuke17/5fad8...dcb04c86e8 but that doesn't seem to have done anything 20:32
was that at least a somewhat-close-to-correct thought? 20:33
jnthnwrthngtn MasterDuke: Is $block[1] a QAST::Stmts node? If so I'd expect that to help but... 20:36
MasterDuke yep
yep 20:37
jnthnwrthngtn MasterDuke: I think the problem is actually about positional captures
github.com/Raku/nqp/blob/master/sr...s.nqp#L834 20:38
In there we never emit a QAST::Stmts around things afaict, and that is probably a problem 20:39
MasterDuke ah, i'll try some keyboard flailing around there
i did look at all the cases of `rxtype<subcapture>` in rakudo's actions and they all had a `:node($/)` 20:40