Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes. Set by lizmat on 24 May 2021. |
|||
00:50
MasterDuke joined
|
|||
MasterDuke | timo: that's cool. has it taught you anything about how to make the jit, moarvm, nqp, or rakudo better? | 00:54 | |
05:25
[Coke]_ joined
05:27
[Coke] left
06:19
Voldenet left
06:20
Voldenet joined
|
|||
patrickb | I'm really impressed by this. This will prove really useful next time I'll need to look at jitty stuff. | 06:53 | |
Where does the code live? | |||
07:32
sena_kun joined
|
|||
timo | patrickb: looking at it might currently make people's heads explode. i'll need to do some drastic refactoring before it can live anywhere, really | 07:52 | |
patrickb | Yeah, definitely work on the head explosions before prime time. Would be a real shame. | 07:53 | |
08:00
sena_kun left
|
|||
timo | i'm going through the gdb docs and finding more interesting stuff, too | 08:05 | |
like, it was required to build an unwinder to make the stack not b0rk after seeing a jitted frame | 08:06 | ||
but then, i can also build an unwinder that shows moar-level stack frames on the "regular" stack, like, above the MVM_interp_run for example | |||
(above MVM_interp_run is physically where a jitted function lives on the C stack) | 08:07 | ||
it's kinda weird to have the regular moar bytecode frames above MVM_interp_run and below the jitted frame, since that would imply that the jit returns past all the moar frames and into the interpreter | 08:08 | ||
it's also weird to have the moar bytecode frames below MVM_interp_run since that implies that when MVM_interp_run returns it goes into the uppermost moar bytecode frame | |||
on the other hand, you could imagine a moar bytecode frame to be a series of calls into the bytecode switch/case inside MVM_interp_run, if you kind of twist your mind into thinking "goto NEXT" inside the MVM_interp_run being like a return into the bytecode frame | 08:10 | ||
the unwinding api allows me to essentially create frames out of thin air with whatever semantics i like, and the frame filter api and frame decorator api let me change how they are displayed | 08:11 | ||
for example, there's a "elided" frame display option where stack frames get indented, which is meant for inlined frames. we do have inlined code! | 08:12 | ||
also, there's the stack records and dispatchers that all want in on the gdb integration action | |||
patrickb | timo: btw. did you hear of my debugger frontend project I'm currently working on? Based on Terminal::Widgets. I want to make moar return checksums for all code it sees (so I can verify the debugger sees the same code moar does) and opt-out-y have it return the source code itself, so you don't have to go through the why-u-no-see-my-sources-they-r-right-there-dance. | 08:14 | |
timo | i did not hear! | 08:15 | |
please feel free to elaborate lots if you like | |||
patrickb | So there is lots of nice stuff in the debugger API if we only manage to bend enough to make moar fit the model. :-P | 08:16 | |
Idea is to annotate the byte code while compiling with the zstd compressed source and a (line-Endung/bom/whatever normalized) checksum of the source and then have the moar debugger API return it. | 08:18 | ||
need to get zstd into core for that to work. | |||
Some flag can prevent the source embedding for enterprise reasons. | 08:19 | ||
The debugger can then just happily work off of the source it gets via the API. | |||
(get-the-source-from-disk will still be supported) | 08:20 | ||
Also I need to get a Pity impl into moar. current plan is to steal from node-pty. | 08:21 | ||
afk for a bit | |||
timo | pity is a library to get the nastiness that is PTY handling somewhat doable? | ||
patrickb | Jup | 08:46 | |
Basically the only bit that lib gives me is platform independently creating a master and slave pty handle, nothing more. | 08:48 | ||
github.com/microsoft/node-pty/tree/main/src the stuff in the Unix and ein folder I want to steal | |||
s/ein/win/ | |||
timo | are you stuck on something? since i built some amount of the debugserver, i can surely be of assistance for a thing or two | 08:49 | |
patrickb | I know. I'd be glad to have you available as I'll surely have questions. Currently I'm happily progressing though. :-) | 08:50 | |
My guess is, that the most difficult part starts, once I have the frontend working. I've seen many cases of it failing (deadlocking, breakpoints not hitting, ...). Ironing those out will be a challenge. But your current gdb work might be invaluable for that. \o/ | 08:53 | ||
timo | well, there's of course also value in going through WIP stuff in case i spot something where you're on the way to a dead-end, or where something better is available, etc etc | 08:56 | |
patrickb | Currently I'm working on Terminal::Widgets bits (that lib needs quite some more work, but I don't have to do all of it, japhb is also working on it). I plan to relatively soon move on to a minimal syntax highlighter, then the Moar bits. So it'll be some time before I dig into moar. | 09:02 | |
If you want to have a look, the current WIP is at git.sr.ht/~patrickb/app-moarvm-tuidebugger | |||
It's a challenge to get it working though. I have patches in Terminal-Widgets and I think some other libs as well. Not all are upstream yet. | 09:04 | ||
Also it's really a WIP, not much of the debugger logic itself is there yet. | 09:05 | ||
timo | that's fair | ||
it'll be a little tricky even with "source is put along with the compiled output" when there's a bunch of compilation-at-runtime and runtime-at-compile-time shenanigans in a typical more-complex program | 09:06 | ||
patrickb | I do plan for it to be epic though. :-D That's why I want to also tackle PTY, highlighting and source retrieval. | 09:07 | |
timo | aye | 09:08 | |
i feel like we never did anything fancy enough with the debughelpers concept | 09:09 | ||
lizmat | patrickb timo for valid Raku source code, there's now RakuAST based syntax highlighting in RakuAST::Deparse::Highlight | ||
timo | essentially, "invoke some random code object via the remote protocol" | ||
patrickb | I'm kind of aware. In my naive world view, I'll just add an anotation to each comp unit with the source. In the ideal case I'll do it for run time compile units as well and if RakuAST is there deparse to generate source. | ||
lizmat: I fear that approach is unfeasible for the debugger setting. I need to parse code I don't have any control over. | 09:11 | ||
lizmat | well, don't pass the :unsafe flag then | ||
it will inhibit "use" for modules, "constant" and BEGIN blocks | 09:12 | ||
patrickb | But will it always succeed even with syntax errors or otherwise unpassable code? | ||
lizmat | nope, it won't, the code will need to be parseable | ||
timo | "throwing a helpful exception" is one kind of success | 09:13 | |
patrickb | Code that uses stuff from modules and doesn't look into use statements is not valid syntax. :-( | 09:14 | |
My dream is to one day have a RakuAST based grammar that is forgiving (i.e. it'll always or most always parse to the end) and does not run begin/use/... I think I even made a ticket for it somewhere. | 09:16 | ||
lizmat | that approach has other failure modes. as modules can change the grammar of the parser | 09:17 | |
patrickb | lizmat: Don't get me wrong, your highlighter is a nice thing and solves the problem for the raku documentation. I just think it's unfeasible for the broader use case. | 09:18 | |
lizmat | so not handing "use" may cause parse errors later | ||
patrickb | that's why it needs to be forgiving (i.e. needs to be able to iron over unknown stuff) | 09:19 | |
lizmat | that's going to be hard if a module exports an infix:<foo> | ||
timo | or a term that looks like a sub name | 09:20 | |
lizmat | just saying :-) | ||
fwiw, I think a mode that would really disallow any I/O etc, "a real safe mode" would make more sense | 09:21 | ||
but maybe that's even harder to achieve, especially if you don't want to exclude NativeCall | 09:22 | ||
timo | really aggressive sandboxing with seccomp and whatnot could be A Thing, but definitely not a cakewalk | 09:23 | |
patrickb | I think debugger I'm working on is the smaller issue. The real challenge is Rakudoc support on raku.land. (I think that's a really pressing issue.) For that to work we need a way to extract the Rakudoc (including Rakudoc that references and annotates code) from the entire ecosystem, including potentially evil code. | 09:24 | |
lizmat | well, we could start with modules that are in the zef ecosystem | 09:26 | |
at least we know who uploaded them | |||
patrickb | We should aim for an approach that is inherently safe. | 09:27 | |
Could we maybe run use, but prohibit BEGIN? And then have a white list of BEGIN blocks to run for modules that modify the parser? | 09:29 | ||
Won't save us from a bad actor hijacking a whitelisted module, though. | 09:31 | ||
lizmat | remember that the mainline of a used module is a very large BEGIN block :-) | ||
patrickb | So allowing use, but prohibit BEGIN is not a thing? | 09:32 | |
This really is a bit of a nightmare. | 09:33 | ||
How realistic is a "real safe mode"? | 09:36 | ||
Is disallowing NativeCall an issue? | |||
timo | well, with NativeCall you have a direct, one-step way to running "shellcode" ;) | 10:01 | |
i.e. arbitrary machine code you supply, or to jump to arbitrary memory locations | |||
okay, maybe two-step. our heap and stack is probably marked noexecute, but of course there's always ROP | 10:02 | ||
patrickb | I'd say we have to disallow NativeCall. I just wondered because lizmat said "especially if you don't want to exclude NativeCall". | 10:38 | |
11:58
sugarbeet left,
sugarbeet joined
12:50
vrurg left,
vrurg joined
12:55
vrurg left
13:00
vrurg joined,
vrurg left
13:01
vrurg joined
13:07
vrurg left
13:08
vrurg joined
13:20
[Coke] joined
13:22
[Coke]_ left
15:14
MasterDuke left
17:15
[Coke] left,
[Coke] joined
18:05
vrurg left,
vrurg joined
18:08
sena_kun joined
20:00
JRaspass left,
JRaspass joined
21:53
[Coke] left
21:56
[Coke] joined
22:06
sena_kun left
23:06
camelia left
23:10
MasterDuke joined
|