Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
jnthnwrthngtn And about 3...NFG is defined in terms of Unicode properties; I think "combining characters" is a slightly hand-waving term, but I imagine anything you'd consider that is covered by the grapheme cluster rules, which are in unicode.org/reports/tr29/#Grapheme...Boundaries 00:01
NFG is "just" giving negative codepoints to things outside of NFC that form a single grapheme cluster
00:02 reportable6 left 00:05 reportable6 joined
jnthnwrthngtn I guess the more nuanced answer to 2 is "maybe there is but not trivially". 00:06
sleep &
japhb Ooof. Well that was a depressing dose of reality. :-P 00:18
Hmmm, let me take a step back. I'm trying to implement an editable text buffer with undo/redo capability. However "insert $new-string at $pos" is an operation that must turn into "replace $some-chunk between ($pos - $some-chunk.chars) and $pos with ($some-chunk ~ $new-string)" because of renormalization across the join boundary. 00:22
So I'm trying to figure out how to determine what $some-chunk will need to be so I can create the undo and redo records. 00:23
00:46 ggoebel joined 01:36 SmokeMachine left 01:43 frost joined 01:44 SmokeMachine joined 02:42 ggoebel left 03:49 JimmyZ joined
JimmyZ Maybe a good read: Different JIT Compilations in a Meta-tracing JIT Compiler Framework arxiv.org/pdf/2011.03516.pdf 03:50
jnthnwrthngtn, nine, brrt ^ 03:51
05:43 JimmyZ left 05:50 JimmyZ joined 06:03 reportable6 left, reportable6 joined 07:19 MasterDuke left, discord-raku-bot left, leont left, Nicholas left 07:30 MasterDuke joined, discord-raku-bot joined, leont joined, Nicholas joined 08:02 root___ joined, JimmyZ left, root___ is now known as JimmyZ 08:05 JimmyZ left, JimmyZ joined 08:52 JimmyZ left 08:53 ggoebel joined
MasterDuke interesting. i just got a slightly different segv. the spesh candidate we got out of a spesh slot here is fubar github.com/MoarVM/MoarVM/pull/1561...b707eR6455 09:13
so i set a watchpoint on that spesh slot and it gets overwritten in MVM_spesh_add_spesh_slot 09:15
but how is that possible? MVM_spesh_add_spesh_slot should be sticking things on the end, not overwriting 09:16
gist.github.com/MasterDuke17/1c009...40b922c9d2 so info from rr 09:27
*some
ha! thought i'd see what asan had to say. this was its suggestion "...Dissassemble the provided pc to learn which register was used.". that's pretty drastic, didn't think i'd have to take apart my computer to figure this bug out 09:58
jnthnwrthngtn moarning o/ 10:06
Nicholas \o
MasterDuke any weekend-inspired fixes for the spesh optimization inconsistencies? 10:19
jnthnwrthngtn No; needed to do family things and try to rest a bit.
So barely hacked on anything (an hour or so on a private fun project...) 10:20
Not too much use working on it here at the office either, 'cus it turns out that I don't see the inconsistencies here after the various patches I worked on last time.
MasterDuke huh. do you see any difference with clang vs gcc on your office computer? 10:24
10:25 linkable6 left, evalable6 left
jnthnwrthngtn Hm, didn't occur to me to try that. But given I *did* see the difference here, and now don't, and it looks timing related in various ways, I'm sorta suspecting that this is a 6-physical core Xeon chip and home is a 16-physical core (I think? or 32?) AMD chip 10:27
(Well, "now don't" as in "my patch set eliminated them on this machine")
MasterDuke cpu architecture and/or concurrently debugging, the best kinds 10:31
nine jnthnwrthngtn: 16 cores, 32 threads 10:40
jnthnwrthngtn Yeah, sounds right :) I'm not sure there is a 32/64 version :) 10:41
nine Not of Ryzen. You can get up to 64 cores (!) in an AMD Threadripper 10:42
MasterDuke but they don't have a zen3 threadripper yet, right? 10:44
nine AFAIK not. It's still Zen2
dogbert17 jnthnwrthngtn: do you want something super easy to do while drinking your coffee? 10:45
t/spec/S06-advanced/dispatching.rakudo.moar (Wstat: 0 Tests: 6 Failed: 0) 10:46
TODO passed: 1-2
dogbert17 is drinking coffee
btw, I tried to set MVM_HASH_RANDOMIZE=0 in src/moar.h and that seemed to make the gist run 'fast' all the time (tried running it about 15 times) 10:58
10:58 Altai-man joined
Altai-man did anyone remember fixing an issue with `No such method 'CALL-ME' for invocant of type 'Int'`? Not so recent Blin shown the Curry module to fail with this, but it passes on latest for me. Not sure if race or not. 11:09
tellable6 2021-10-03T00:47:49Z #raku <[Coke]> altai-man - is github.com/rakudo/rakudo/issues/4314 closable?
Altai-man [Coke], not yet. :(
11:28 linkable6 joined
dogbert17 Altai-man: FWIW, github.com/rakudo/rakudo/issues/4543 mentions some CALL-ME related changes 11:53
Altai-man ah yes, and the author is the same, I thought I saw the nick somewhere! dogbert17++
12:03 reportable6 left 12:05 reportable6 joined 12:27 evalable6 joined
jnthnwrthngtn Phew, think I finally found/fixed the problem that led my callwith work to pile up doomed dispatch programs (which never ever succeeded) 12:54
Nicholas nine: it compiles - ship it 13:44
OK, so new-disp-nativecall does fail some spectests on ppc64
which IIRC master did not
oh, but x86_64 fails (at least) one test in the same way 13:46
identical fails! 13:47
Geth MoarVM: 7fe8a0d94d | (Jonathan Worthington)++ | src/disp/program.c
Correct compilation of resume inits

When we have a dispatch program that works through multiple levels of resumption, each level may set up one or more new future resumptions. We take care to emit dispatch resumptions that are set up innermost first, however that must be done taking into account the resumption levels. Otherwise, we can end up reading resume init state of the original resumptions at the wrong time.
13:48
13:56 frost left 14:04 kjp left
nine Nicholas: what fail is that? 14:11
14:20 lizmat_ joined 14:21 TempIRCLogger left, TempIRCLogger joined 14:24 lizmat left 14:27 kjp joined 14:29 lizmat_ is now known as lizmat, lizmat left 14:30 lizmat joined
MasterDuke jnthnwrthngtn: btw, SmokeMachine had a rakuast question over in #raku "Hi! I'm starting playing with RakuAST and I'm not being able to add a custom pass on it... should it be working? usercontent.irccloud-cdn.com/file/...image.png" 14:32
jnthnwrthngtn MasterDuke: Ah, they asked me elsewhere, and I answered :) 14:40
MasterDuke cool
thought i'd do something a bit different today, went on a shortish hike up, down, and around cooper's hill, where they hold the famous cheese rolling competition. it really is quite steep, not at all surprised at the number of injuries sustained 14:45
Geth MoarVM: b535721442 | (Jonathan Worthington)++ | src/disp/resume.c
Only resume innermost resumable dispatch

The rule were meant to be:
  * Disregard any non-resumable dispatch encountered
  * If we encounter a resumption that itself sets up further resumptions,
   it is logically part of some base dispatch, so keep looking
... (8 more lines)
14:47
lizmat and yet another Rakudo Weekly News hits the Net: rakudoweekly.blog/2021/10/11/2021-...-patterns/ 14:49
jnthnwrthngtn Yay :) 14:53
Nicholas nine: paste.scsys.co.uk/595994 15:04
nine Ooh spectests. I haven't run any yet 15:17
It's odd that they would be affected by NativeCall changes at all. Maybe my branch is just too far behind master 15:18
Nicholas oh. in which case, I'm dissapointed by how few you've broken. Try harder! :-)
yes, possibly it's just a bit "last week"
nine Now why would adding the dispatcher-track-unbox-int syscall and MVMDispOpcodeUnboxInt op lead to segfaults in the expression JIT when running the profiler on an empty program? Even when neither syscall nor op are ever used? 15:40
jnthnwrthngtn nine: I had great "fun" by forgetting to update labels.h 15:41
And then having the computed goto jump to quite random places :)
nine Thankfully the compiler actually warned me about that! 15:42
jnthnwrthngtn Lucky you, mine didn't! 15:43
nine A bit indirectly though, with an "unused" warning.
Does the order of those ops matter?
jnthnwrthngtn The order in the labels.h must match the order in the enum since the opcode indexes the array 15:45
But otherwise, I don't believe so; the order they're in is mostly organizational (grouping similar things) 15:46
nine Ok. Didn't know that the order matters, but did it right anyway 15:48
walk_tree (tc=0x4c3e70, tree=0x7ffff02025e0, traverser=0x7ffff6dbbf90, node=-153370240) 15:49
jnthnwrthngtn Phew, think I'm much of the way through github.com/rakudo/rakudo/issues/4542 15:50
MasterDuke while people are here, how is possible that MVM_spesh_add_spesh_slot is overwriting a slot? that's not normal/expected, right? 15:55
jnthnwrthngtn Sounds wrong to me 16:00
We used to have some sp_findmethod that would write into a spesh slot, but it's long gone 16:01
MasterDuke github.com/MoarVM/MoarVM/blame/mas...ze.c#L1564 does 16:05
nine jnthnwrthngtn: oh, it's not the addition of dispatcher-track-unbox-int or MVMDispOpcodeUnboxInt after all! I had sp_nativecall already added to src/core/oplist but nowhere else 16:13
jnthnwrthngtn MasterDuke: Does what? That line is updating a spesh operand? 16:14
MasterDuke yeah. github.com/MoarVM/MoarVM/blame/mas...ze.c#L1437 16:17
jnthnwrthngtn I don't see the connection with spesh slots, it's the index of the spesh candidate that we're pre-selecting 16:23
Ah, but maybe you need this to refer to a spesh slot containing the candidate now?
In that case I'd expect it'd need to add the candidate to a spesh slot and update the index there, but... 16:24
MasterDuke yep
jnthnwrthngtn Oh no, it's OK, I think -1 is used as the "no spesh candidate linked" value 16:25
MasterDuke github.com/MoarVM/MoarVM/pull/1561...f498fbe07a
jnthnwrthngtn So spesh slots start at 0
MasterDuke oops, github.com/MoarVM/MoarVM/pull/1561...75f94f528e
and github.com/MoarVM/MoarVM/pull/1561...310077fbe9 16:26
jnthnwrthngtn Hm, that looks better 16:29
Only issue may be that I think the sslot type in oplist implies unsinged, but we stick a -1 in there
May have to leave that change out
nine Apparently the spect test failures all have the same reason: the check for a custom dispatcher gives bogus results when junctions are involved: 16:30
nqp::isconcrete(my $custom-dispatcher := nqp::decont(nqp::how_nd($code).find_method($code, 'CUSTOM-DISPATCHER')))
The weird thing about this is, that this ^^^ is exactly the same code as the check for CALL-ME with just the names replaced
MasterDuke i printed out the values in the sp_runbytecode_* ops in interp.c and there are -1s there 16:33
e.g., here github.com/MoarVM/MoarVM/pull/1561...b707eR6391 16:34
16:45 Geth left 16:46 Geth joined, RakuIRCLogger joined
jnthnwrthngtn OK 16:50
Hm, then I don't immediately see what's missing/wrong 16:51
17:43 Altai-man left 17:56 squashable6 left 17:59 squashable6 joined 18:02 reportable6 left
Nicholas jnthnwrthngtn: ASAN finds new-disp-callwith most uninteresting 18:47
19:04 reportable6 joined
nine jnthnwrthngtn: I figured out my spec test failures wrt CUSTOM-DISPATCHER. Need to pass a :no-fallback to nqp::how_nd($code).find_method($code, 'CUSTOM-DISPATCHER', :no-fallback). Maybe a good idea for the search for CALL-ME as well? 19:35
github.com/rakudo/rakudo/blob/mast...s.nqp#L623 19:43
20:03 patrickb joined
japhb jnthnwrthngtn: I really appreciate that you mark in the commit description when you add a big block doc comment to a commit (as in the rakudo commit earlier). Helps me not miss the deeper info. :-) 20:15
jnthnwrthngtn nine: Ah, yes, probably it is 20:23
nine: Should I or will you? 20:24
[Coke] me: finally give up and install a rakudo binary so I can get dayjob scripts running again. Try another build from source... it jfw. (can still reproduce the --debug == concurrent NC failures) 20:53
No idea what unblocked. 20:54
[Coke] will try another, non-debug build
debugserver.c(1010) : warning C4700: uninitialized local variable 'tc' used 20:59
seems like a legit warning. 21:02
21:40 patrickb left
[Coke] so at some point during a rebuild, a 'git clean -xdf' started dying complaining it couldn't remove nqp-m.exe from the nqp build dir. I think once that started happening, any builds were going to be a problem. I rebooted, whatever had the file locked had let go, and I got a successful build. 22:05
so I'm guessing some variant of that was what was giving me trouble earlier. 22:06
jnthnwrthngtn Urgh, sounds like 22:10
[Coke] sorry for the wild goose chase! 22:14
I tried a few things to free the lock, but no luck. (said it had itself open, couldn't see nqp-m in the task listing) 22:15
(hence reboot) 22:20
jnthnwrthngtn Well, I solved a debug server bug on Windows while I was in the VM, so it wasn't all for nothing :) 22:30
[Coke] nice! 22:48
japhb jnthnwrthngtn: Looking at the link you sent yesterday for MVM_nfg_is_concat_stable -- any particular reason lines 450..453 (github.com/MoarVM/MoarVM/blob/d11b...fg.c#L450) shouldn't be hoisted up between 433 and 444? It looks like when the MVM_unicode_relative_ccc if succeeds, it throws away all the other work in the surrounding block. 23:02
The summary of my review of the comments in that function is: only the last codepoint of the previous string could change during string concat renormalization *unless* the joining point is surrounded by region indicators. 23:05
Interestingly, reviewing the source for MVM_string_concatenate, I *think* consumed_a (count of graphemes consumed from first string) can only be 0 or 1 in that code. So either the region indicators comment isn't really a problem, or MVM_string_concatenate never got updated for region indicators. 23:45