Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:40 frost joined
timo idea of the day: when we are eq_I-ing two values where one is known to come from fastbox_bi_ic, we could emit like sp_eq_Ii instead, which could possibly take shortcuts 00:48
01:03 reportable6 joined 01:19 MasterDuke left 02:19 linkable6 left, evalable6 left 02:20 evalable6 joined 03:39 squashable6 left, evalable6 left, tellable6 left, notable6 left, bloatable6 left, nativecallable6 left, quotable6 left, sourceable6 left, coverable6 left, statisfiable6 left, greppable6 left, unicodable6 left, benchable6 left, shareable6 left, bisectable6 left, reportable6 left, committable6 left, releasable6 left 03:40 squashable6 joined, notable6 joined, reportable6 joined 03:41 sourceable6 joined, shareable6 joined, committable6 joined, bloatable6 joined, unicodable6 joined, nativecallable6 joined 03:42 greppable6 joined, releasable6 joined, evalable6 joined 04:20 linkable6 joined 04:41 benchable6 joined 04:42 coverable6 joined, statisfiable6 joined 04:44 frost left, dogbert17 left, gfldex left 04:45 frost joined, dogbert17 joined, gfldex joined 05:42 tellable6 joined, quotable6 joined 05:54 japhb left 06:02 reportable6 left 06:05 reportable6 joined 06:23 frost left
Nicholas good *, #moarvm 06:48
06:49 japhb joined 08:57 MasterDuke joined
jnthnwrthngtn moarning o/ 09:47
timo gratings 09:49
Nicholas \o 09:51
jnthnwrthngtn
.oO( cheese ones, I hope... )
 10:02
timo the "turn a given/when with integer matchers into a jumplist" optimization; that would be for rakuast rather than for moarvm, right? 10:03
jnthnwrthngtn Yes, but also depends on us not allowing junction on the LHS, so probably a 6.e thing 10:13
timo ah, ok 10:15
nine In my effort to debug the expr JIT, I may have made the first step towards real unit tests in MoarVM. At least I've found out that it's rather simple to create a synthetic spesh graph for easier debugging/testing. 10:16
MasterDuke oh! nice
jnthnwrthngtn ooh :)
lizmat jnthnwrthngtn: so I assume you're on the "banning" side of the github.com/Raku/problem-solving/issues/297 argument ?
nine It's just about 50 lines of code and gives me a spesh graph with a single gethow instruction, so I no longer have to look through 50 dumps to find my basic block that contains instructions I don't need
jnthnwrthngtn lizmat: There's been a bit too much to follow in there (and I've not read all of it yet), but I certainly agree that from an optimization perspective, Junction allowed on the LHS is problematic. 10:19
lizmat share the feeling on both observations :-)
jnthnwrthngtn lizmat: I mean, I'd like us to compile $foo ~~ SomeType into nqp::istype, for example.
You can't do that if the LHS should auto-thread 10:20
lizmat so one could argue that, you being architect in residence, we should find out how we can create an alternative to the *.all ~~ Int idiom? 10:21
jnthnwrthngtn Now, in that case one could figure out how to do a dispatcher that looks at if the LHS is a Junction and re-dispatches to istype (well, we can with a bit of internal re-org in MoarVM to make type checking a normal dispatcher)
But jump tables are rather harder; what would we do, emit the code in duplicate for the junction and not junction case?
lizmat yeah, gotcha 10:22
jnthnwrthngtn m: my @a = 1..5; say so all @a >>~~>> Int 10:23
camelia True
jnthnwrthngtn m: my @a = 0, 1.5, 3; say so all @a >>~~>> Int
camelia False
lizmat that is a lot less DWIM than *.all ~~ Int :-) 10:24
m: my @a = 0, 1.5, 3; say so all @a>>.ACCEPTS(Int)
camelia False
lizmat m: my @a = 0, 1, 3; say so all @a>>.ACCEPTS(Int)
camelia False
lizmat hmmm
jnthnwrthngtn Well, the `so` isn't really needed in most cases anyway 10:25
One other way of looking at it is @a.infer ~~ Int
Where `infer` is a method that "infers" the element type by finding the loosest common ancestor type 10:26
(We may want a better name)
ah, sorry, tightest common :)
timo narrow like we have for numeric types? 10:27
jnthnwrthngtn Sorta but narrow is about the value
lizmat hmmm... I like .infer
jnthnwrthngtn This is kinda the opposite
Well, actually kinda not in that we want the narrowest type that covers all of the cases
Unlike `.narrow` we're not giving back a value though, but rather a type 10:28
timo fair
lizmat do we have an op to easily check for the narrowest common type? 10:29
jnthnwrthngtn No. And we'll have to decide how we're going to define it also. 10:30
Only on types in the MRO, or on roles too?
s/we'll/we'd/ # this is surely still speculation :)
But I think the algorithm is to have a "current working type" which is initialized to the .WHAT of the first element 10:31
And then you go through checking each element against that, and upon finding a contradiction, apply whatever algorithm we pick to find the narrowest covering type 10:32
lizmat yeah, that's what I'm working on right now :-)
jnthnwrthngtn If it's just MRO then you just search the MRO backwards until they diverge
lizmat yup, that's going to be the first prototype
jnthnwrthngtn But I suspect folks would like `[1, 2.3, 5]` to infer Real, not Cool.
lizmat hmmmm yes 10:33
what do we infer on an empty list ? Mu ? 10:34
Any?
jnthnwrthngtn Dunno, we sometimes talk about Nil as our kind of bottom type
MasterDuke i've sort of wondered about a syntax sugar that allows for `sub foo(each-element-is-an-Int-even-if-not-explicitly-typed-as-such @a) { ... }; foo([3, 4])` as sugar for the `sub foo(@a where @a.all ~~ Int) { ... }` 10:35
i'd hoped that allowing coersives to type an array would do what i want, but they don't 10:36
jnthnwrthngtn No, I think coercion types and structural types are probably different axes.
moon-child MasterDuke: can kind of do that. sub f(Array[Int]()) 10:37
jnthnwrthngtn I guess the coercion approach kinda has the benefit that while yes, you make a copy, the result is then properly typed for O(1) check if you pass it on inwards to other things 10:38
moon-child 'structural types' is kind of where/subset, no? In this case sub f(@x where .all ~~ Int) 10:39
jnthnwrthngtn (That also want an Array[Int] and will coerce if needed)
MasterDuke moon-child: but you have to use a $ variable, right? e.g., `sub foo(Array[Int]() $a) { ... }`
moon-child MasterDuke: yes, or sigilless
jnthnwrthngtn moon-child: Yes; I just meant that if we wanted to sugar up such things, it'd want its own syntax, not to re-use the coercion one. 10:40
moon-child nods 10:41
jnthnwrthngtn But on balance we might be better making Array[Int]() $foo style things somehow go nicer with having the sigil
MasterDuke yeah, that's possible, but i'd question who would be able to figure it out when first approaching the language. it just doesn't seem very DWIM
jnthnwrthngtn (The @ sigil, I mean)
MasterDuke yep
jnthnwrthngtn I guess there's kind of a third thing here though 10:42
moon-child how about: sub f(Array[Int]() $@x) ?
jnthnwrthngtn Oh, no, maybe not. Array[Int]() is not about coercing elements, just producing a typed array, so it's fine. 10:43
Arguably Array is over-specifying, though
MasterDuke i think if we're going to come up with new syntax or sugar, it can't have a $ in it at all
at least not in the name of the variable 10:44
`sub foo($Int @a)` perhaps 10:45
moon-child I was imagining that that syntax would declare a variable named @x, but pun its sigil to $ for the purpose of the type constraint. A la my @x; die unless $@x ~~ Array[Int](). I think it might still make sense to support that, but I agree 'Array' overspecifies 10:46
jnthnwrthngtn `Int @a` is another spelling of `@a of Int` so maybe we could also look in that direction for syntax.
(that is, an alternative trait) 10:47
Nicholas nine: what (if anything) is left to do on the new-disp nativecall front?
moon-child Another thing that would be nice is to specify both k and v types for hashes 10:48
jnthnwrthngtn Ah, in an upgrade situation? Yes.
lizmat moon-child: but you can? 10:49
m: my Int %h{Str}
camelia ( no output )
MasterDuke while we're talking about signatures and rakuast and junctions, would the very existence of junctions throw a complete wrench into my pondering about a multi routine syntax that allow you to create one multi for the case of two arguments where you implement the body the same just with the types switched?
lizmat MasterDuke: feels like: yes ? 10:50
jnthnwrthngtn MasterDuke: No, there was already even a speculated syntax for that.
lizmat ah?
moon-child m: sub f(Num %h{Num}) { ... }
camelia 5===SORRY!5=== Error while compiling <tmp>
Shape declaration is not yet implemented; please use whitespace if you meant something else
at <tmp>:1
------> 3sub f(Num %h7ā5{Num}) { ... }
expecting any of:
shape declaratioā€¦
MasterDuke oh!
moon-child lizmat: I just meant that whatever sugary coercive syntax is devised for positionals, it should also apply to associatives cf above 10:51
jnthnwrthngtn MasterDuke: I couldn't find it in the synopses, but github.com/perl6/std/blob/master/STD.pm6#L1900 10:55
10:56 linkable6 left, evalable6 left
MasterDuke thanks, i was just about to ask where in the synopses it was because i was looking and didn't find it 10:56
10:58 linkable6 joined
nine Nicholas: I'm pretty sure there's a commit message or two that are only stubs yet. And a bit of rebasing fixups 11:19
Nicholas: in other words, boring book keeping work that I procrastinate by investigating this 1 in 10000 race condition 11:20
Nicholas I was also wrong in my assessment of what can go in the future - only the nativecallinvoke OP can go. The other two near it are still used, but the nqp:: mapping for each is a different(ish) name, and I got confused. I seem to be easily confused. 11:24
11:41 bisectable6 joined
MasterDuke i find the template compiler errors a bit opaque. `Require reference for write operand $0 (sp_rebless)` 11:47
12:02 reportable6 left
MasterDuke well, given that a spectest with `MVM_JIT_EXPR_DISABLE=1` is faster than without, maybe it's better to not have templates for not very hot ops anyway. anyone have any further thoughts on github.com/MoarVM/MoarVM/pull/1584 ? 12:13
nine approved an earlier version of it, but i've added some more ops and moved the scset(code|obj) implementations out of emit.dasc and just made them c functions that the interpreter and the jit call 12:14
assuming that's what people prefer, i'll squash those scset(code|obj) commits to reduce blame noise 12:15
jnthnwrthngtn I think having them as C functions is much lower risk of errors 12:17
And they do enough that there call overhead is not so high 12:18
MasterDuke yeah, i don't have a preference, i just left both version in for comparison 12:19
timo is there something we can do so that TypedCArray's AT-POS could get away without a Proxy? 12:34
i'm looking at a place where there's 264 calls, for each of the calls it does one call each to raku-invoke, raku-call, and raku-meth-call-resolved, which take 36.23%, 22.74%, and 15.68% respectively 12:37
so i guess it's never hitting what's already in the inline cache?
the stats in the spesh log for the code installed for Proxy's FETCH method (which forwards to whatever the proxy in question has assigned to its &!FETCH) show only total hits and hits for one callsite, nothing logged at all? 12:42
i don't see a "can't inline" anywhere for that frame, and also no places where it is inlined successfully 12:45
MasterDuke why does just the TypedCArray's AT-POS have to be 'is rw' anyway? 12:47
timo as opposed to the others that are "is raw"? 12:50
MasterDuke yeah 12:51
timo not sure 12:53
MasterDuke related question github.com/rakudo/rakudo/commit/6c...#r59698175 12:55
jnthnwrthngtn, lizmat: ^^^
lizmat answered 12:56
MasterDuke m: my $a = (^3).SetHash; dd $a; $a.values[2] = 5; dd $a 13:00
camelia SetHash $a = SetHash.new(1,2,0)
SetHash $a = SetHash.new(1,2,0)
MasterDuke m: my $a = (^3).BagHash; dd $a; $a.values[2] = 5; dd $a 13:01
camelia BagHash $a = (1=>1,2=>1,0=>1).BagHash
BagHash $a = (1=>1,2=>1,0=>5).BagHash
MasterDuke feel like i'm missing something 13:03
lizmat the order of values is indeterminate? so values[2] would be setting a different element each time ? 13:23
BagHash is a Hash ? 13:24
MasterDuke m: my $a = (^3).BagHash; dd $a; $a.values[2] = Nil; dd $a; # guess this makes sense 13:28
camelia BagHash $a = (1=>1,2=>1,0=>1).BagHash
Use of Nil.Int coerced to empty string
in block <unit> at <tmp> line 1
Impossible coercion from 'Nil' into 'Int': method Int returned an instance of Str
in block <unit> at <tmp> line 1
jnthnwrthngtn timo: Sounds odd that it's not getting inline cache hits; do you have a small repro showing it?
lizmat jnthnwrthngtn: github.com/rakudo/rakudo/pull/4626 # implementation of .infer
MasterDuke m: my $a = (^3).SetHash; dd $a; $a.values[2] = Nil; dd $a; # guess this i mean makes sense
camelia SetHash $a = SetHash.new(0,1,2)
SetHash $a = SetHash.new(0,1)
jnthnwrthngtn lizmat: Ah nice, so roles too 13:30
lizmat yup
too bad the bytecode is too large to inline :-( 13:45
will think about that after some fresh air& 13:46
jnthnwrthngtn I...suspsect it's still going to perform rather better than the junction approach? :) 13:47
13:48 Geth left, Geth joined
dogbert17 nine: if you need a break from the '1 in 10000 race condition' problem new ideas wrt the hyper/sprintf bug are always welcome :) 14:53
14:56 [Coke] left 14:58 evalable6 joined 14:59 [Coke] joined 15:03 reportable6 joined 15:16 squashable6 left
jnthnwrthngtn Cool, I now have a branch in Rakudo where use of nqp::attrinited is completely removed in favor of the container descriptor approach. It passes spectest except one test (which I know how to fix, and is a place we were doing something a bit dubious anyway) 15:16
And it also fixes github.com/rakudo/rakudo/issues/4624 along the way 15:17
(Although I'm primarily doing this to avoid having to live through the special hell of tracking auto-viv in the EA... :))
MasterDuke is that an immediate optimization, or more for unlocking future optimizations?
ah, for EA, nice 15:18
jnthnwrthngtn MasterDuke: If anything in the immediate it'll be a tiny slow-down
But it'll permit us to get rid of attribute auto-viv
Entirely
Which means smaller JIT code for all the attribute accesses
And a branch gone in the interpreted case too 15:19
Well, just unspecialized access, in fact.
OK, we don't have enough tests, it turns out. :) 15:20
oh hah, the way Agrammon breaks (I just picked a big app to throw at the branch to see what happens) is a case that I had in my impl notes to handle and forgot :D
We sure don't have enough spectests, then :)
MasterDuke i wonder if it would make sense to add something like `make m-integration-tests` where we run a bunch of contributed large programs, just as a step in between a spectest and a full blin run (since i think most developers don't have blin setup locally (i don't)) 15:24
jnthnwrthngtn Setting up a daily blin in theory "only" needs somebody to do a bit of setup work 15:33
Yay, fixed that bit of it also. 15:54
And written more tests, but gotta go to a meeting now, bbl 15:55
nine dogbert17: I think doing fewer things in parallel is the healthier approach :) 16:06
16:26 [Coke]_ joined 16:28 [Coke] left, [Coke]_ is now known as [Coke]
dogbert17 nine: indeed :) have you managed to get the error to occur more often? 16:51
dogbert17 does an incantation which will hopefully summon brrt 16:54
MasterDuke does comma open heap snapshots? 16:55
nine dogbert17: no, but that's no problem, since I can see the error clearly in the generated code. And thanks to my synthetic spesh tree, I can easily debug the codegen. Which currently means improving the expr JIT's log output to include operands 17:15
I just realized that with synthetic spesh graphs, I should be able to easily set up a better test case with a loop around gethow and a decont (to make it explode on a NULL) and another thread that's flipping that HOW pointer between NULL and a more useful pointer. 17:20
jnthnwrthngtn MasterDuke: Not yet, we're like half way (or so) into implementing heap snapshot analysis in Comma though :) 17:21
MasterDuke cool
nine Executing code needs a bit of setup as well, but hopefully not _that_ much
17:28 evalable6 left, linkable6 left 17:31 linkable6 joined
dogbert17 nine: impressive debug work 17:33
MasterDuke jnthnwrthngtn: to race `for @lines.skip.kv -> $idx, $iter is rw {` it should just be `race for @lines.skip.kv.race -> $idx, $iter is rw {`, right? 17:37
jnthnwrthngtn MasterDuke: You don't need the inner `.race` there (that's only needed to change default degree and batch size) 17:38
MasterDuke: However, there's a problem: .race/.hyper don't yet support parallelizing when there's a multi-arg block (unless somebody implemented it and I missed it :))
MasterDuke it's not using any more than 1 cpu 17:39
oh, ah
jnthnwrthngtn So probably better to for @lines.skip.paris
18:03 reportable6 left 18:31 evalable6 joined
lizmat jnthnwrthngtn github.com/rakudo/rakudo/pull/4626/files now takes a 9x as fast path if all types are identical 18:35
@a where .infer ~~ Int is about 3x as fast as @a where .all ~~ Int (for ^10) 18:40
jnthnwrthngtn Well, nice to have a carrot as well as a stick... :) 18:48
Geth MoarVM/new-disp-nativecall-libffi: 32 commits pushed by (Stefan Seifert)++, (Nicholas Clark)++
review: github.com/MoarVM/MoarVM/compare/b...10c7d7100a		 18:52
MoarVM: niner++ created pull request #1595:
New disp nativecall
lizmat jnthnwrthngtn I was thinking that maybe a .isa-all(type) method would actually be better performance-wise
as it could short-circuit, which .infer can not
so a signature of an array with only Ints, would become "@a where .isa-all(Int)" 18:53
(which is another 15% faster for the all-Int case) 18:54
and much faster for non-matching cases
nine new-disp-nativecall is ready for review :) Thanks again to Nicholas++
lizmat nine++ 18:55
nine: perhaps a description in the PR would be nice?
nine lizmat: indeed. Done! 19:02
19:03 reportable6 joined
MasterDuke lizmat: does .isa-all work with coercion types? e.g., `@a where .isa-all(Int())` 19:14
lizmat it uses nqp::istype under the hood 19:15
Nicholas and all the pipelines are "fail" because they try to build MoarVM new-disp-nativecall-libffi with nqp master and rakudo master? (when it would need the new-disp-nativecall branches for both) ?
lizmat MasterDuke: yeah, looks like it does :-) 19:16
MasterDuke nice
lizmat my @a = "1","2","3"; sub a(@a where .isa-all(Int())) { }; a @a # works
it doesn't actually coerce the array though 19:17
but you didn't expect that, did you MasterDuke ?
MasterDuke no 19:18
but i wonder if it'd be too magical to have a `.all-could-be(::T)` that automatically does a `.isa-all(T())` 19:19
changing topics. the MVM_oops seen in andinus's Fornax module when .racing Cairo.write-png stems from github.com/rakudo/rakudo/blob/mast...on.pm6#L54 and github.com/rakudo/rakudo/blob/mast...OW.nqp#L90 19:22
and github.com/timo/cairo-p6/blob/mast....pm6#L1091 19:23
istr i (i think it was me) wrapping some other metamodel code (i think it was something about types) in a lock.protect. is that appropriate here? 19:29
ah, github.com/rakudo/rakudo/commit/58...5228903e82 was the start (there were some subsequent related commits) 19:30
nine Nicholas: yes 19:34
19:43 Geth left, TempIRCLogger left 19:45 RakuIRCLogger left, lizmat left 19:47 lizmat joined 19:48 Geth joined
MasterDuke one could probably get CI to test those branches by modifying github.com/MoarVM/MoarVM/blob/mast...ml#L17-L18 to be e.g., 'rev-<sha1 of the last commit of their branch>-github.com/rakudo/rakudo.git' 19:51
19:51 Kaipi joined
Nicholas I thought that something like that would be possible, but the "problem" then is that *that* commit doesn't want to get merged. 19:52
MasterDuke github.com/MoarVM/MoarVM/blob/mast...est.pl#L28
right, you'd have to remove it after CI was green but before the merge
a bit annoying 19:53
19:53 childlikempress joined
Nicholas I *think* it might work "reliably" if our entire CI setup had a way to say "at least this commit SHA1" meaning that if master is beyond that SHA1, CI uses master. But if master is a parent of that commit, use that commit 19:53
(and I realised part way though, if master and that commit are different branches, safer to use master and make lots of red) 19:54
MasterDuke i'm not sure if the clone+lock dance are actually needed in this case, but i've been running the fornax code in a loop for a while now with gist.github.com/MasterDuke17/ec8d7...fad1eb0a9b applied and no MVM_oops yet 19:55
20:00 Kaiepi left, discord-raku-bot left, moon-child left
MasterDuke lizmat: gonna add a .isa-none? 20:03
lizmat hmmmm
isn't that !isa-any ? 20:04
I guess it could be added for completeness sake? 20:09
nine MasterDuke: I think that patch is neither enough, nor efficient. It's not enough because the lock doesn't cover all the code that accesses the hash. It's possible that 2 threads compete to add something to the hash. After the first wrote, it releases the lock at which point the second thread starts writing. Meanwhile the first thread tries to read and boom 20:13
MasterDuke: it's also not efficient since you always lock (on every read). That's where the clone + lock idiom comes in. Cloning and replacing the lookup hash atomically protects concurrent readers. Locking protects against concurrent writers. 20:14
lizmat MasterDuke: PR for isa-none added 20:18
20:26 childlikempress is now known as moon-child
jnthnwrthngtn I don't think isa-* makes sense because .isa is only really about inheritance, but this would presumably about roles too. 21:13
MasterDuke like-*? could-be-*? matches-*? 21:14
is-or-does-*? 21:15
naming things is hard
21:16 squashable6 joined
timo MasterDuke: that code from EnumHOW could be changed so that it constructs the hash and only at the end binds it into the %!value_to_enum slot, then it wouldn't concurrently resize the same hash. it would race to install the hash, but that's not a problem since every thread would end up with the same hash and then the losers would just have theirs garbage collected 21:17
lizmat all-is-type / any-is-type / none-is-type ? 21:19
MasterDuke add_enum_value needs a fix too, right?
timo if you do that in parallel, maybe you deserve to crash :) :) 21:20
MasterDuke does replacing the hash need to be wrapped in nqp::scwb(dis|en)able()? 21:21
timo ah, quite possibly 21:23
otherwise your code would repossess the enum just by being the first to use it 21:24
and now i see that nine also already pointed that out 21:26
jnthnwrthngtn The boolean methods feel a bit low-wattage to me, tbh. I like an `infer`-alike because it'd have other applications, in the same way that .all is.
s/is/does/ 21:27
MasterDuke gist.github.com/MasterDuke17/ec8d7...fad1eb0a9b updated. this version has also run in a loop for a long time (much longer than it usually takes to get an MVM_oops) without problem 21:28
lizmat jnthnwrthngtn: I created the bool methods because they can short-circuit, which may be of importance with larger lists ? 21:29
21:30 discord-raku-bot joined
lizmat in any case, I've made the PRs, *I* think they make sense, but I could well live without them 21:35
I have no personal beef in them, so if they are all rejected, that's fine by me :-)
MasterDuke how/where do the frames created by allocate_specialized_frame() get freed if running with --full-cleanup? 22:57
23:44 dogbert17 left
jnthnwrthngtn MasterDuke: Stack unwinding if they're still on the call stack (same as the unspecialized ones) 23:49
Geth MoarVM: MasterDuke17++ created pull request #1596:
Use alloca in MVM_string_memmem_grapheme32str...		 23:50
jnthnwrthngtn If heap allocated or heap promoted, maybe in MVM_frame_destroy
MasterDuke jnthnwrthngtn: thanks. i think i may be been barking up a wrong tree though. i saw a ton of leaks from allocate_specialized_frame() in heaptrack, but later i realized it had actually aborted before raku finished cleaning up because of the problem with collecting locked mutexes 23:52
jnthnwrthngtn Ah 23:54
The allocate_specialized_frame and allocate_unspecialized_frame was just a split of allocate_frame into the two cases to avoid dupe conditions, so shoudla been low risk. The special return changes do restructure things a bit with regard to where things are freed (but pretty sure that's still unmerged) 23:55
'night o/ 23:58

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!