Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:02 reportable6 left 00:03 reportable6 joined 00:44 patrickb left 00:45 patrickb joined, squashable6 left 00:46 squashable6 joined 00:56 patrickb left 03:34 [Coke] left 03:55 [Coke] joined 05:51 squashable6 left, committable6 left, releasable6 left, greppable6 left, quotable6 left, linkable6 left, reportable6 left, evalable6 left, notable6 left, benchable6 left, nativecallable6 left, statisfiable6 left, sourceable6 left, coverable6 left, tellable6 left, unicodable6 left, shareable6 left, bisectable6 left, bloatable6 left 05:52 unicodable6 joined, linkable6 joined, coverable6 joined, statisfiable6 joined, benchable6 joined 05:53 releasable6 joined, squashable6 joined, nativecallable6 joined, bloatable6 joined, greppable6 joined, shareable6 joined, quotable6 joined 06:11 frost joined 06:12 frost left 06:30 frost joined 06:49 leont left 06:52 leont joined, tellable6 joined, bisectable6 joined 06:53 sourceable6 joined 07:05 reportable6 joined 07:32 CIAvash joined
CIAvash jnthnwrthngtn: It may not matter now, but I saw logs.liz.nl/moarvm/2021-11-09.html#10:23 through lizmat's `infer` PR. And that was one of the first things that came to my mind, but it doesn't work properly if the topic has Lists, because hyperoperators go inside the Lists. So my other solutions were github.com/Raku/problem-solving/is...-966016474 07:40
07:52 committable6 joined, notable6 joined
nine m: my $i = 2**64 - 1; sub foo(uint64 $ui) { say $ui; }; foo($i) 08:08
08:08 AlexDaniel joined
camelia Cannot unbox 64 bit wide bigint into native integer
in sub foo at <tmp> line 1
in block <unit> at <tmp> line 1
 08:08
08:08 psydroid joined
nine The Algorithm::Evolutionary::Simple test failure is simply this ^^^. The bug was just hidden before 08:09
And when I fix autounbox to actually call MVM_repr_get_uint and the perfectly valid 64 unsigned value gets through, the result is: -1 08:26
And Algorithm::Evolutionary::Simple still fails with "Cannot unbox negative bigint into native unsigned integer". 08:34
my $buf = Buf[uint64].new(2**64-1); 08:41
m: my $buf = Buf[uint64].new(2**64-1);
camelia Cannot unbox 64 bit wide bigint into native integer
in block <unit> at <tmp> line 1
nine This one's caused by there being no bindpos_u. So instead, we emit bindpos_i which needs a native int64, which is why we emit a decont_i and not a decont_u. 08:47
08:54 evalable6 joined 09:27 Colt joined
lizmat on the Ecosystem::Archive failure seems to occur with waiting for the header of a Cro::HTTP::Client.get($URL) 09:31
after it doing X times successfully
nine lizmat: does it occur with MVM_JIT_DISABLE=1? 09:34
lizmat where X is between 27 and 34 09:36
lizmat tries
indeed it does not, so it appears
yup, that makes it not happen 09:38
lizmat just received word that David H. Adler has passed away 09:42
aka dha on IRC
logs.liz.nl/search.html?nicks=dha&...ies-pp=500 09:43
www.facebook.com/ginarsnape/posts/...2275758366 09:46
09:57 RakuIRCLogger joined 09:58 Geth joined
nine lizmat: :( 10:02
I get the impression that our unsigned handling is just broken on every level and in every part of the codebase. So I said, that there's no bindpos_u. How hard can it be to add one? 10:05
Well MVM_VMArray_bind_pos checks if the register type fits to the slot type. So far for unsigned slots it _required_ signed registers (because there's only bindpos_i). Changing that reveals all the places where we do a manual bindpos_i where bindpos_u would be more appropriate. 10:06
Of course fixing those creates bootstrap issues, but well. They are not insurmountable. But thenQAST::MASTOperations.add_hll_unbox('', $MVM_reg_uint64, -> $qastcomp, $reg {
This unbox still tries to generate a smrt_intify op. Which does not exist after new-disp 10:07
That we've made it so far anyway just shows how often we silently switch to signed types and just never generate unsigned unboxing
But then, even if smart_intify was still there, we'd actually generate code that deconts, intifies and then coerces from int to uint. We don't even consider the possibility that the thing may unbox to uint directly 10:10
dogbert17 nine: I can offer bugs free of rabbit holes (lie) :) 10:27
lizmat is glad nine is looking at the yack in the room 10:34
nine I guess we'll need an nqp-uintify to accompany nqp-intify. The difference being that it'd be looking for a UInt coercer method instead of Int. I think for the rest we'd get by with a little signed/unsigned sloppyness as it doesn't affect the result. 10:35
10:39 linkable6 left 10:42 tealecloud joined, linkable6 joined 10:44 linkable6 left
nine Yes, that gets me through NQP's build 10:45
And rakudo compiles with this pretty bit in the array::intarray postcircumfix:<[ ]> candidate: !! SELF.^array_type.^unsigned ?? nqp::bindpos_u(nqp::decont(SELF),$pos,assignee) !! nqp::bindpos_i(nqp::decont(SELF),$pos,assignee) 11:00
lizmat that's in Rakudo or NQP ? 11:01
nine Rakudo 11:03
Of course that needs a real fix, or it'd tank performance
With this, my $buf = buf64.new(2**64-1); at least succeeds. But of course dd $buf then gives me Buf[uint64] $buf = Buf[uint64].new(-1) 11:05
Which isn't completely wrong. But doesn't make debugging easier either
lizmat yeah, I've learned to not expect unsigned values to work into the sign bit :-( 11:06
jnthnwrthngtn moarning o/
lizmat jnthnwrthngtn o/
more like a mourning, alas :-(
jnthnwrthngtn So I see :-( 11:07
nine bindpos_u seems to work. But of course to actually get an unsigned to bind in that test, we also need an atpos_u 11:29
lizmat well... if you get that to work, I can do the necessary changes in Rakudo I think :-) 11:32
11:35 Altai-man joined 11:46 linkable6 joined 12:02 reportable6 left 12:03 reportable6 joined 12:08 linkable6 left
Geth MoarVM: b744f8640d | Altai-man++ (committed using GitHub Web editor) | docs/release_guide.md
Update MoarVM release guide according to the actual release process (#1599)

Thank you, Oleksandr Kyriukhin!		 12:10
Altai-man for the record, I can reproduce nativecall-related issues, e.g. TCP::LowLevel gives me the same error, while it installs fine on 2021.10. 12:16
nine Of course the [ ] candidate for reading doesn't actually use atpos_i. It uses atposref_i. To get an unsigned version of that we'll need an unsigned version of NativeRef with unsigned accessors 12:40
lizmat and the yack gets bigger :-( 12:41
nine And that needs an UIntLexRef type alongside the existing IntLexRef. And all the special handling those types get 12:42
I estimate that fixing this properly is a month long project in itself 12:44
12:49 frost left
timo does raku run fine on mips machines? 12:53
(disregarding that the mips machines i have seen are all super low on both disk and ram)
Nicholas IIRC I was able to build it in 32 bit mode on the mips machines on the GCC compiler farm 12:54
but in 64 bit mode, they only have enough RAM for NQP to build.
timo how much ram do they have? 12:55
rakudo needs what, like a gig? for the core setting?
turn off spesh and it'll get a little smaller i think
Nicholas cfarm.tetaneutral.net/machines/list/ -- the winner is "gcc22" at 1980 MB 12:56
and no, I hadn't tried without spesh
timo wow, that was not enough?
Nicholas IIRC no
Meg
but in *64* bit mode it was not enough
32 bit was OK 12:57
timo it might be a while still before routers make their web admin interfaces as cro apps 13:05
13:05 frost joined
Nicholas well, it's building the setting that is the hog - if the build system is larger and distributes a completed tree then life is easier 13:06
and also, why would the router want to run 64 bits?
nine The build system can easily be larger and much, much faster. Like your ordinary household variety Ryzen desktop. Remember that MoarVM bytecode files are architecture independent and entirely portable! 13:08
13:08 linkable6 joined
Nicholas I was about to type "but we don't actually test that" and then I remembered stage0 bootstrap in NQP ... 13:11
13:57 CIAvash left 14:03 frost left 14:21 linkable6 left 15:22 linkable6 joined 15:56 tealecloud left 17:09 evalable6 left, linkable6 left 17:12 evalable6 joined
dogbert17 I reported github.com/MoarVM/MoarVM/issues/1607 so it doesn't fall through the cracks 17:16
nine mÖ use nqp; nqp::dispatch("boot-syscall", "dispatcher-register", "boom", nqp::getattr(-> $capture { die "boom" }, Code, <$!do>)); sub foo() {}; BEGIN &foo does role :: { method CUSTOM-DISPATCHER(--> str) { "boom" } }; { foo(); CATCH { default { } } }; dd $! 17:42
m: use nqp; nqp::dispatch("boot-syscall", "dispatcher-register", "boom", nqp::getattr(-> $capture { die "boom" }, Code, <$!do>)); sub foo() {}; BEGIN &foo does role :: { method CUSTOM-DISPATCHER(--> str) { "boom" } }; { foo(); CATCH { default { } } }; dd $!
camelia AdHoc $! = X::AdHoc.new(payload => "boom")
nine This ^^^ is the LibraryCheck issue. For some reason, having an exception in a dispatcher sets $!, despite the docs claiming that $! only gets set by try
jnthnwrthngtn m: { die "oops"; CATCH { default { } } }; say $! 17:45
camelia oops
in block <unit> at <tmp> line 1
jnthnwrthngtn Not specific to dispatchers, but rather CATCH seems to be doing it 17:46
nine Huh....that's the first thing I tried but I only got Nil. 17:47
Ah, just got confused by my own output 17:48
So far I've investigated two issues and both point at pre-existing problems that may be hard to get rid of :? 17:49
:/
Well the unsigned issues are definitely hard to get rid of
Oh, but if when it doesn't have anything to do with dispatchers why is it only a problem now? NativeCall's setup has thrown this exception previously and CATCH was there as well 17:51
And $! is not set within that CATCH block. That's weird, too 17:52
18:02 reportable6 left
nine Apparently it's been this way since basically forever. Changing that is surprisingy simple (just delete 4 LOC), spec test clean and gets rid of the problem 18:15
This was too easy. Makes me nervous
lizmat doesn't see a commit message yet, is Geth down ?
nine Haven't committed yet. Should I push straight to master? 18:16
lizmat if you haz a gist somewhere, I'll run the spectest :-) 18:17
nine gist.github.com/niner/2fcd520e859b...ea17ccfceb 18:19
lizmat anything particular I should test apart from the spectest ? 18:28
nine Whatever's important to you :) 18:29
lizmat what is this supposed to fix again? the NativeCall issues ? 18:31
nine Those "Missing serialize REPR function for REPR VMException (BOOTException)" errors appearing in Blin output 18:33
Like in MUGS::Games
jnthnwrthngtn But if it's been like this forever, then...what changed to make it a problem now? 18:41
nine That's the mystery 18:43
jnthnwrthngtn :) 18:45
Time to go home and make dinner o/ 18:47
18:48 Altai-man left
nine At least that "MoarVM oops in spesh thread: Spesh inline: could not find appropriate pred to update" was relatively easy to fix as well 18:50
lizmat I don't see any additional breakage and cannot install MUGS 19:03
starts eating memory like crazy when testing Cro::TLS
so I guess there is no reason to not do this, nine :-) 19:04
MasterDuke nine: doubt they're of any use, but my old WIP commits around adding *_u things github.com/MasterDuke17/MoarVM/tre...re_missing and github.com/MasterDuke17/MoarVM/tre...bindattr_u 19:09
lizmat nine: you will be glad to see that today's test-t shows that csv-ip5xs went from 1.4 to 0.870 since the merge, and csv-ip5xs-20 from 15.58 to 5.47 19:13
that's Quite an improvement
eh, 5.32 actually 19:14
nine Oooh...closing in on csv-parser 19:15
And only a factor 13 away from pure csv-test-xs
19:48 evalable6 left 19:50 evalable6 joined 20:05 reportable6 joined
lizmat does anybody have an explanation for the following observed behaviour: 20:26
I start up about 10 start blocks, and keeping the promises in an array 20:27
then waiting for the promises in turn to be kept
each start block does a .race on a list of values with the highest possible :degree and appropriate :batch size
in the beginning it uses all of the processors (8+8), but as soon as a promise is kept, the CPU usage also goes down 20:28
which makes me think that somehow each start block gets some CPU allotted, and that doesn't change even if there are no other threads working anymore 20:29
so instead of using all possible CPU until the last race is run, it just starts taking less and less CPU 20:30
making the process slower than it could be
timo how does the snapper look? did you try Log::Timeline?
lizmat gist.github.com/lizmat/f7720fd13ce...a3cd4d2e66 20:33
note that you can see it go up to ~85%, and then drop off to about 35% after that 20:36
added a gtraph of CPUY usage 20:39
timo i wonder what exactly runs on the five affinity threads. are that the splitters for the race pipeline perhaps? 20:42
lizmat switching it to a .race(:1batch) doesn't change it 20:45
timo ok i imagine each stage of the pipeline is counted as one "task" in terms of tasks completed 20:47
20:47 colemanx left
Geth MoarVM: 44c88da7cf | (Stefan Seifert)++ | src/spesh/manipulate.c
Fix invalid pred chain caused by MVM_spesh_manipulate_split_BB_at

MVM_spesh_manipulate_split_BB_at splits a basic block such as the later part becomes a new basic block. However it neglected to update the successors of the original basic block to point to the newly formed one instead. This caused spesh to blow up when inlining. Fix by replacing all references to the original in the successors' pred arrays by references to the new one.		 20:48
nine This is the fix for Imlib2 20:49
lizmat nine++
timo it'd be pretty cool if we had something that can visualize what exactly a scheduler is doing, with more details about the tasks than we can just get right now 20:50
maybe Log::Timeline, maybe something else
nine DB::SQLite may be a JIT issue
21:34 squashable6 left 21:48 [Coke] left 21:50 [Coke] joined
jnthnwrthngtn Hm, I was looking into if Spreadsheet::XLSX has a memory leak (well, probably one of its deps) so wanted to run creating a spreadsheet in a loop. 21:57
And soon got a SEGV
(Inside the Raku LibXML module, in a dispatch program) 21:58
timo oof, not good
jnthnwrthngtn Doesn't happen with spesh disabled 21:59
Does appear to leak though
22:11 linkable6 joined 22:18 patrickb joined 22:36 squashable6 joined
MasterDuke speaking of leaks, i compiled CORE.c with --full-cleanup under heaptrack a couple days ago and got almost zero leaks. there was 500bytes leaked, but 300 was from pthread (which i assume is the spesh thread not being cleaned that nine has some wip on). the other 200 was from MVM_frame_move_to_heap 22:53
but overall a pretty good showing

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!