Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes.
Set by lizmat on 24 May 2021.
00:50 MasterDuke joined
MasterDuke timo: that's cool. has it taught you anything about how to make the jit, moarvm, nqp, or rakudo better? 00:54
05:25 [Coke]_ joined 05:27 [Coke] left 06:19 Voldenet left 06:20 Voldenet joined
patrickb I'm really impressed by this. This will prove really useful next time I'll need to look at jitty stuff. 06:53
Where does the code live?
07:32 sena_kun joined
timo patrickb: looking at it might currently make people's heads explode. i'll need to do some drastic refactoring before it can live anywhere, really 07:52
patrickb Yeah, definitely work on the head explosions before prime time. Would be a real shame. 07:53
08:00 sena_kun left
timo i'm going through the gdb docs and finding more interesting stuff, too 08:05
like, it was required to build an unwinder to make the stack not b0rk after seeing a jitted frame 08:06
but then, i can also build an unwinder that shows moar-level stack frames on the "regular" stack, like, above the MVM_interp_run for example
(above MVM_interp_run is physically where a jitted function lives on the C stack) 08:07
it's kinda weird to have the regular moar bytecode frames above MVM_interp_run and below the jitted frame, since that would imply that the jit returns past all the moar frames and into the interpreter 08:08
it's also weird to have the moar bytecode frames below MVM_interp_run since that implies that when MVM_interp_run returns it goes into the uppermost moar bytecode frame
on the other hand, you could imagine a moar bytecode frame to be a series of calls into the bytecode switch/case inside MVM_interp_run, if you kind of twist your mind into thinking "goto NEXT" inside the MVM_interp_run being like a return into the bytecode frame 08:10
the unwinding api allows me to essentially create frames out of thin air with whatever semantics i like, and the frame filter api and frame decorator api let me change how they are displayed 08:11
for example, there's a "elided" frame display option where stack frames get indented, which is meant for inlined frames. we do have inlined code! 08:12
also, there's the stack records and dispatchers that all want in on the gdb integration action
patrickb timo: btw. did you hear of my debugger frontend project I'm currently working on? Based on Terminal::Widgets. I want to make moar return checksums for all code it sees (so I can verify the debugger sees the same code moar does) and opt-out-y have it return the source code itself, so you don't have to go through the why-u-no-see-my-sources-they-r-right-there-dance. 08:14
timo i did not hear! 08:15
please feel free to elaborate lots if you like
patrickb So there is lots of nice stuff in the debugger API if we only manage to bend enough to make moar fit the model. :-P 08:16
Idea is to annotate the byte code while compiling with the zstd compressed source and a (line-Endung/bom/whatever normalized) checksum of the source and then have the moar debugger API return it. 08:18
need to get zstd into core for that to work.
Some flag can prevent the source embedding for enterprise reasons. 08:19
The debugger can then just happily work off of the source it gets via the API.
(get-the-source-from-disk will still be supported) 08:20
Also I need to get a Pity impl into moar. current plan is to steal from node-pty. 08:21
afk for a bit
timo pity is a library to get the nastiness that is PTY handling somewhat doable?
patrickb Jup 08:46
Basically the only bit that lib gives me is platform independently creating a master and slave pty handle, nothing more. 08:48
github.com/microsoft/node-pty/tree/main/src the stuff in the Unix and ein folder I want to steal
s/ein/win/
timo are you stuck on something? since i built some amount of the debugserver, i can surely be of assistance for a thing or two 08:49
patrickb I know. I'd be glad to have you available as I'll surely have questions. Currently I'm happily progressing though. :-) 08:50
My guess is, that the most difficult part starts, once I have the frontend working. I've seen many cases of it failing (deadlocking, breakpoints not hitting, ...). Ironing those out will be a challenge. But your current gdb work might be invaluable for that. \o/ 08:53
timo well, there's of course also value in going through WIP stuff in case i spot something where you're on the way to a dead-end, or where something better is available, etc etc 08:56
patrickb Currently I'm working on Terminal::Widgets bits (that lib needs quite some more work, but I don't have to do all of it, japhb is also working on it). I plan to relatively soon move on to a minimal syntax highlighter, then the Moar bits. So it'll be some time before I dig into moar. 09:02
If you want to have a look, the current WIP is at git.sr.ht/~patrickb/app-moarvm-tuidebugger
It's a challenge to get it working though. I have patches in Terminal-Widgets and I think some other libs as well. Not all are upstream yet. 09:04
Also it's really a WIP, not much of the debugger logic itself is there yet. 09:05
timo that's fair
it'll be a little tricky even with "source is put along with the compiled output" when there's a bunch of compilation-at-runtime and runtime-at-compile-time shenanigans in a typical more-complex program 09:06
patrickb I do plan for it to be epic though. :-D That's why I want to also tackle PTY, highlighting and source retrieval. 09:07
timo aye 09:08
i feel like we never did anything fancy enough with the debughelpers concept 09:09
lizmat patrickb timo for valid Raku source code, there's now RakuAST based syntax highlighting in RakuAST::Deparse::Highlight
timo essentially, "invoke some random code object via the remote protocol"
patrickb I'm kind of aware. In my naive world view, I'll just add an anotation to each comp unit with the source. In the ideal case I'll do it for run time compile units as well and if RakuAST is there deparse to generate source.
lizmat: I fear that approach is unfeasible for the debugger setting. I need to parse code I don't have any control over. 09:11
lizmat well, don't pass the :unsafe flag then
it will inhibit "use" for modules, "constant" and BEGIN blocks 09:12
patrickb But will it always succeed even with syntax errors or otherwise unpassable code?
lizmat nope, it won't, the code will need to be parseable
timo "throwing a helpful exception" is one kind of success 09:13
patrickb Code that uses stuff from modules and doesn't look into use statements is not valid syntax. :-( 09:14
My dream is to one day have a RakuAST based grammar that is forgiving (i.e. it'll always or most always parse to the end) and does not run begin/use/... I think I even made a ticket for it somewhere. 09:16
lizmat that approach has other failure modes. as modules can change the grammar of the parser 09:17
patrickb lizmat: Don't get me wrong, your highlighter is a nice thing and solves the problem for the raku documentation. I just think it's unfeasible for the broader use case. 09:18
lizmat so not handing "use" may cause parse errors later
patrickb that's why it needs to be forgiving (i.e. needs to be able to iron over unknown stuff) 09:19
lizmat that's going to be hard if a module exports an infix:<foo>
timo or a term that looks like a sub name 09:20
lizmat just saying :-)
fwiw, I think a mode that would really disallow any I/O etc, "a real safe mode" would make more sense 09:21
but maybe that's even harder to achieve, especially if you don't want to exclude NativeCall 09:22
timo really aggressive sandboxing with seccomp and whatnot could be A Thing, but definitely not a cakewalk 09:23
patrickb I think debugger I'm working on is the smaller issue. The real challenge is Rakudoc support on raku.land. (I think that's a really pressing issue.) For that to work we need a way to extract the Rakudoc (including Rakudoc that references and annotates code) from the entire ecosystem, including potentially evil code. 09:24
lizmat well, we could start with modules that are in the zef ecosystem 09:26
at least we know who uploaded them
patrickb We should aim for an approach that is inherently safe. 09:27
Could we maybe run use, but prohibit BEGIN? And then have a white list of BEGIN blocks to run for modules that modify the parser? 09:29
Won't save us from a bad actor hijacking a whitelisted module, though. 09:31
lizmat remember that the mainline of a used module is a very large BEGIN block :-)
patrickb So allowing use, but prohibit BEGIN is not a thing? 09:32
This really is a bit of a nightmare. 09:33
How realistic is a "real safe mode"? 09:36
Is disallowing NativeCall an issue?
timo well, with NativeCall you have a direct, one-step way to running "shellcode" ;) 10:01
i.e. arbitrary machine code you supply, or to jump to arbitrary memory locations
okay, maybe two-step. our heap and stack is probably marked noexecute, but of course there's always ROP 10:02
patrickb I'd say we have to disallow NativeCall. I just wondered because lizmat said "especially if you don't want to exclude NativeCall". 10:38
11:58 sugarbeet left, sugarbeet joined 12:50 vrurg left, vrurg joined 12:55 vrurg left 13:00 vrurg joined, vrurg left 13:01 vrurg joined 13:07 vrurg left 13:08 vrurg joined 13:20 [Coke] joined 13:22 [Coke]_ left 15:14 MasterDuke left 17:15 [Coke] left, [Coke] joined 18:05 vrurg left, vrurg joined 18:08 sena_kun joined 20:00 JRaspass left, JRaspass joined 21:53 [Coke] left 21:56 [Coke] joined 22:06 sena_kun left 23:06 camelia left 23:10 MasterDuke joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!