timotimo i think so 00:12
01:36 colomon joined 02:35 colomon joined 02:49 ilbot3 joined
hoelzro I feel like I'm at the end of this bug, but I can't quite get it 03:34
I don't think I really understand the relationship between bytecode loading and deserialization
06:53 domidumont joined 06:58 domidumont joined 07:24 FROGGS joined 08:19 zakharyas joined 08:36 brrt joined
brrt JimmyZ: why do you say that (MoarVM shall have tests), I don't know of that? 08:42
timotimo hello brrt :) 08:47
brrt hi timotimo
scroll shadows in high contrast mode *facepalm* 08:48
not well thought out this is
timotimo: will you be at FOSDEM perchance?
timotimo nope :(
"scroll shadows"?
brrt yes, in applications, if you reach the end 08:51
of scrolling
too bad
timotimo oh 08:56
so with high contrast, they aren't actually changed at all so they just lower the overall contrast and legibility of things, eh? 08:57
brrt the shadows don't change, no
the rest of the stuff does
which is handy sometimes :-)
how are things? still plans? 08:58
timotimo hm? what kind of plans?
brrt to hack :-) 09:01
timotimo ah, right 09:03
well, i'm behind one day on the weekly, but i want to contribute a little patch to krita (an open-source drawing/painting program) just so i don't have my head in perl6 all day every day 09:04
brrt :-) 09:06
that's well enough
brrt is in the finishing stages of his thesis now :-)
i found a limiting case of python
a loop of 50 lines long.. i can't tell which block belongs where 09:07
timotimo to be fair, braces don't help much there, do they?
brrt so i'm resorting to counting indentation, this sucks
timotimo i have a vim plugin that colors areas by indentation depth; it may not be high-contrast enough for you, though
brrt braces would help; they always have a corresponding other brace 09:08
so i can have emacs tell me where the closing brace responds to
corresponsds
anyway 09:09
i haven't really hacked on moarvm htese last few weeks
timotimo there ought to be something in emacs that'll give you that same feature with python 09:13
brrt hmmm 09:14
probably, yes :-)
timotimo i have no clue if there is. but it ought to be made if there isn't :)
09:19 kjs_ joined 09:20 kjs_ joined
timotimo humpf. my simple five-line patch idea just turned into what amounts to a swim through potentially a big amount of code >_> 09:35
i'm not thrilled
JimmyZ brrt: not really me, jnthn++ said it 2 years ago, IIRC. 09:39
brrt well, it hasn't become a priority yet 09:40
i think testing unit-ish things like string ops, that may be plausible
JimmyZ yes, so, it will
timotimo yeah
brrt why are you so sure? the hardest part are the most difficult to test, it seems
stuff like gc, concurrency etc 09:41
thats already tested by nqp
FROGGS I think that that running nqp's testsuite is good enough
brrt although i'd agree with the sentiment that we would like a bit more
FROGGS you dont even need to build it, you could even check out stage0 and run the tests
brrt but thats still for me personally to figure out, how to test the jit better
JimmyZ like, test some code whether it is jitted well enough after we changed the spesh/jit code 09:47
or s/it/it still/
brrt that is a good idea; however i think that can live in the nqp testsuite 09:48
moar doesn't really have a separate personality next to nqp 09:49
timotimo agreed 09:50
i've sat next to a discussion about giving moarvm a little bootstrapped moar+nqp of its own for build system purposes before 09:51
brrt hmmpf 09:52
timotimo with that nqp copy, a freshly-built moar could run its own test suite which would then be written in nqp
brrt turtles will have to stop somewhere
JimmyZ we did have some test in moarvm and only for moarvm around 2013
timotimo :) 09:53
JimmyZ it uses nqp to generate some moarvm only code.
like fib
something like use QAST
10:10 leont joined 10:24 virtualsue joined, zakharyas joined 10:37 kjs_ joined 11:11 kjs_ joined 11:20 TimToady joined 12:19 brrt joined 12:42 virtualsue joined 13:03 brrt joined
jnthn hoelzro: Bytecode loading triggers a "we're being loaded" frame to be interpreted, which somewhere calls a deserialize op. It's actually up to that code when exactly deserialize takes place. So the two things are rather decoupled. 13:15
I don't think that we'll have a Moar test suite that covers the same set of things the NQP test suite does, since the NQP test suite does an increasingly good job of covering all the ops. 13:16
timotimo especially thanks to pmurias' work
jnthn Yes, pmurias++ for that.
We may have some tests for MoarVM that stress GC, or maybe optimization-related tests. I imagine we'll write them in NQP; since they're stress rather than sanity tests, it doesn't matter hugely if we can't run them until we build an NQP. :) 13:17
hoelzro jnthn: so a .moarvm file contains bytecode, which is arranged in the structures described in the documentation. One of the frames is the deserializer, which calls deserialize with a string in that bytecode file's heap; is that correct? 13:53
jnthn Used to be
It actually passes the op a null string 13:54
And that means "go find the serialization blob from a special segment in the bytecode file"
Which avoids having to base-64 it to shove it in the strong heap
hoelzro ooooh, I was wondering why there was a null_s in the moar dump
jnthn: which segment is that? 13:55
ah, I see it 13:57
the SC data, it seems
jnthn oh, actually I lied
timotimo i had this random idea: can we annotate the gc_mark functions of our reprs with something that'll put all of them together in a special section in the final binary?
jnthn /* Special frames. */ 13:58
MVMStaticFrame *main_frame;
MVMStaticFrame *load_frame;
MVMStaticFrame *deserialize_frame;
deserialize_frame is run unconditional of the "load mode"
timotimo perhaps that can improve strain on the code cache during GC mark runs?
jnthn load is run if it's loaded in any way other than as the mainline
main is run if it's loaded as the program entry point
timotimo: Not run into such an annotation 13:59
timotimo: Also not sure there's enough gc_mark routines to have huge impact 14:00
timotimo mhm
jnthn (The reasoning is sound, just not sure if it'll amount to measurable.)
timotimo and the gc always has to go through the REPR pointer anyway
probably not
hoelzro jnthn: sorry for all the questions, but could you clarify the difference between an MVMFrame that's loaded as a result of deserialize_frames in the bytecode loading, vs an MVMFrame loaded as a result of deserialize_contexts in SC loading?
timotimo is just spewing random ideas at this point :)
things that come to me during showers, or while falling asleep
hoelzro I'm at my wits' end with this bug, and it all comes down to that
jnthn hoelzro: Confusing terminology is confusing :) 14:01
/* The various static frames in the compilation unit, along with a
* code object for each one. */
MVMStaticFrame **frames;
And
static MVMStaticFrame ** deserialize_frames(MVMThreadContext *tc, MVMCompUnit *cu, ReaderState *rs) {
Those are actually both about *static* frames
deserialize_contexts is about actual MVMFrames 14:02
MVMStaticFrame contains all the static things about a call frame (mapping of lexical names to indexes, the bytecode, etc.)
MVMFrame is an actual invocation record
An MVMFrame points to an MVMStaticFrame 14:03
Does that help clear things up a little?
hoelzro ah ha
yes, it does
they both seem to carry lexical information, though
jnthn Maybe the language used in the code was clearing up
An MVMFrame carries the actual set of values lexicals are bound to 14:04
It doesn't know anything about names, though. It's just a bunch of indexed slots.
MVMStaticFrame carries the metadata on what types of values are in the slots and the name that goes with each slot.
hoelzro what's confusing to me is that deserialize_context itself creates a MVMStaticFrame/MVMFrame pair
so I'm not sure where the MVMFrames from bytecode decoding come in 14:05
jnthn MVMFrames aren't produced by bytecode decoding... 14:06
(There's no MVMFrame mentioned at all in bytecode.c)
hoelzro hmm...let me see if I can find the code that made me think that 14:07
it may make more sense after reading your explanation
jnthn Sure. Feel free to point me at any other bits of code you want me to explain further also.
hoelzro I suppose it was MVM_bytecode_finish_frame 14:08
but that has to do with static frames
jnthn Yes
That's just laziness
hoelzro got it
jnthn (As in, putting off work until we have to do it)
hoelzro well, let me describe my issue, and maybe you can point me in the right direction
jnthn Well, and the name is typing laziness too :P
hoelzro heh =)
basically, if I create an anonymous sub at BEGIN time in a module, that sub cannot make calls to subs in CORE at runtime 14:09
jnthn We probably shouldn't pun on two so easily confused things, except we often get away wiht it...
hoelzro it can make method calls
or qualified sub calls
jnthn Ah...I fixed something like that before for constants...
git log --author=jnthn --grep=constant 14:10
hoelzro in my example, the frame->outer chain is three frames longs; if I remove the BEGIN, it's like six
moar --dump says that the outers chain is longer than I see in practice
jnthn hmm, that don't find it
hoelzro here's the actual RT: rt.perl.org/Ticket/Display.html?id=127089 14:11
jnthn Ah, I was misremembering 14:12
76204fa58a7
Is the Rakudo commit I last touched in this area
I think it's the same kind of issue. 14:13
hoelzro ah ha
jnthn Maybe we shouldn't be teaching BEGIN/role bodies specially
Maybe we should be doing this on *any* code we trigger at compile time.
hoelzro that makes sense to me
jnthn s/teaching/treating/
I'm not sure off hand how hard it will be, other than "it'll be hard because this stuff is always a pain" 14:14
hoelzro =/
well, at least I should be able to fix my issue for now
if not, I know where to find you =)
thanks a ton, jnthn
timotimo thanks jon-a-ton 14:15
hoelzro haha
jnthn :P
jnthn wonders if there isn't a more elegant solution to this kind of problem... 14:17
So far I've only come up with solutions I hate less than what came before them. :)
hoelzro as long as hate is a strictly non-increasing function over time, it's ok =P
14:22 virtualsue joined
timotimo don't want to get stuck in a spiral of hatred 14:24
jnthn
.oO( How does a hate spiral compare with a love triangle? )
 14:27
moritz less edges, more hateful
timotimo probably also has spiders in it
nwc10 more dimensions
a triangle is at best a plane 14:28
15:12 FROGGS joined 15:23 domidumont joined
ilmari gist.github.com/ilmari/2a23f96373163a1fa45b 16:53
Draft MoarVM magic(5) file ^^
hoelzro ilmari++ 16:54
ilmari dunno how relevant all the sizes/counts are
maybe just display the bytecode and serialization versions?
hoelzro I would probably just do that 16:55
jnthn They're nice to have in a sense. 16:56
But maybe I think that 'cus Moar doesn't provide a good way to get at them already :) 16:57
ilmari I notice the bytecode itself doesn't have a version. does moar check for unknown ops at load time? 17:02
jnthn It's part of validation 17:04
We should at some point establish version policy around adding new ops.
Probably some point soonish given we're in a post-6.c world and we'd like to be able to have people easily upgrade their MoarVM without upgrading the things running on it. 17:05
ilmari ah, in get_info()
as long as we add new ops at the end and don't remove existing ones, that would Just Work™ 17:06
jnthn Oh, that direction works already, yes 17:07
It's more that it'd be better to up-front detect accidents like somehow running a newer bytecode file on a Moar that can't handle all the ops in it.
ilmari afaict validation.c does that 17:08
jnthn Yes, with the caveat that we don't validate bytecode for a frame until we actually execute that frame.
So the error can come a good way down the line.
ilmari ah
that's LTA
also, the error message is a bit LTA too: "Bytecode validation error at offset %u, instruction %u:\ninvalid opcode %u" 17:09
jnthn Right. It's not that we don't catch it, but a better versioning policy would let us give a better experience.
ilmari being a) up-front and b) explicit about "your moar is too old" would be MA
jnthn Agree
New ops are pretty rare already, so we could think about making it policy already. 17:10
ilmari well, I added two only the other day
jnthn OK, they're more rare than they maybe were before :)
I think mostly I mean they're uncommon enough that I don't mind the extra developer time overhead of having to bump a number. 17:11
17:11 kjs_ joined
ilmari yeah, but there currently doesn't exist such a number (unless you want to bump the top-level number each time) 17:16
jnthn I'm not sure there's a downside to bumping the top level number for that 17:24
ilmari but then, why the separate serialization version? 17:25
jnthn Because the serializer can be used independent of bytecode files (in fact, we used to store the serialized blob on the string heap) 17:27
18:20 leont joined 18:27 vendethiel joined 18:29 domidumont joined 18:58 kjs_ joined 19:00 FROGGS joined 19:23 leont joined 19:28 virtualsue joined
jnthn Oh noes...I just tried to build Rakudo on my desktop after being away for a while and... msinttypes/inttypes.h is missing 19:36
19:45 zakharyas joined 19:46 virtualsue joined
FROGGS :S 19:50
dalek arVM: 940850e | jnthn++ | Configure.pl:
Correct installation of msinttypes header test.
 19:52
FROGGS jnthn++ # I merged the PR that broke it 19:53
(sorry)
jnthn No worries, wasn't a hard fix
Somewhat curious why things seem slower on this box than before...
FROGGS do you clean your install/? 19:54
jnthn (make test in NQP is up a bit, build times for Rakudo also)
Yeah
FROGGS I mean, because I did not notice the breakage
k
jnthn Ah
You'd get away with it then
FROGGS dunno about the slowdown though
jnthn That's why I didn't notice it on my laptop
ilmari is inordinately amused by the filename moar.magic 19:55
jnthn :D
FROGGS .magic?
where is that from?
geekosaur flips the switch...
FROGGS, libmagic which is the file type detection stuff used by file(1) 19:56
it's driven by config files describing how to identify a given file type
FROGGS I know mime magic... though I did not know about .magic files 19:57
geekosaur (the original file command had a file of "magic numbers" from back when most data files started with a binary value indicating what kind of file it was) 19:58
ilmari FROGGS: gist.github.com/ilmari/2a23f96373163a1fa45b 20:04
FROGGS ilmari: ahh, now I understand
ilmari save that, do file -m moar.magic somefile.moarvm
you'll get nice details
FROGGS awesome
will try
ilmari and geekosaur spotted the reference 20:05
moritz is anybody trying to get moar.magic upstream into file(1)?
or how are such things handled? 20:06
ilmari moritz: I'm planning to submit it to the mailing list once we agree it's in suitable shape
FROGGS: catb.org/jargon/html/magic-story.html
moritz ilmari: great, ilmari++
ilmari hm, wonder if the magic language is flexible enough to extract the HLL name 20:09
but not now, hometime now
should have left half an hour ago, got distracted :-/ 20:10
yes, it can do multiple levels of indirection 20:12
\o/ 20:13
FROGGS CORE.setting.moarvm: MoarVM bytecode (version 5), 7 serialization contexts, 16 extension ops, 10733 frames, 352 callsites, 19921 strings, 5307976 bytes serialized data, 3783540 bytes bytecode, 505272 bytes annotations 20:26
I love it :o)
ilmari++
20:40 zakharyas joined 20:51 FROGGS joined 20:53 kjs_ joined
[Coke] nice! 21:02
21:31 virtualsue joined
ilmari FROGGS: oh, that version doesn't have the serialized data version, the one I have at $ork adds that 21:51
FROGGS now my sword glows blueish
ilmari I was trying to extract the hll name, but that would require looping over the string tables, which I'm not sure it can do 22:01
since the strings are stored as a sequence of <length>,<data> pairs without an index 22:02
FROGGS hmmmm
the string heap, yeah 22:03
ilmari the hll name is just stored as "the Nth string" 22:06
oh, and the length of each string is stored shifted left one, with the bottom bit indicating whether it's latin1 or utf8 22:19
22:42 kjs_ joined
timotimo i'm back 22:53
22:54 virtualsue joined 23:05 kjs_ joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!