github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
00:03 sena_kun joined 00:04 Altai-man_ left 00:26 AlexDaniel left 01:18 Merfont left 02:01 Altai-man_ joined 02:04 sena_kun left
Geth MoarVM: 0racle++ created pull request #1324:
Revert "Declare loop var before loop"		 02:28
03:40 lembark56 joined 03:45 lembark56 left 03:46 lembark87 joined, lembark87 left 03:47 lembark joined
lembark Sorry about that... 03:48
tellable6 2020-06-30T19:10:17Z #raku <MasterDuke> lembark i've had success before using heaptrack to find leaks in moarvm. a regular profile or heap snapshot will give you information at the raku level
lembark Excellent.
I'm trying to convince people to use Raku for ETL and my experience dealing with nr.gz is making me look stupid :-)
Getting heaptrack now... 03:49
I'm not running KDE. Checking the website, there doesn't seem to be a tarball. 03:54
Q: How do I install the thing w/o KDE? 03:55
Q: Is there any way to really force Rakudo to do garbage collection (before I blame this on a leak)?
03:58 Merfont joined
lembark Found it: cloned from githup. 03:59
04:02 sena_kun joined 04:04 Altai-man_ left
Voldenet lembark: use nqp; nqp::force_gc() 04:16
nwc10 good *, #moarvm 04:26
jnthn: I think it might be possible to acchieve (most of) what you wanted for naight threads by using a memory layout of [entry, entry, entry, ... ] [control structure] [metadata, metadata, medadata, ..., sentinel] 04:58
and the public hash point is just a char * pointing to the control structure
timotimo what is a naight thread?
nwc10 and it sits in the middle because it has (potentinally) less alignment restriction than the entries
timotimo: me mistyping naughty 04:59
he meant other threads still accessing hashes that they aren't really supposed to be any more (so data races)
that currently can SEGV
keep the old block around, free it at a safe point
timotimo ah
nwc10 public hash pointer is just
we do lose the ability to lazily allocate storage
but it has the interesting side effect that 05:00
if you (first) convert all the fixed strings in the Unicode table from discrete char * constants in the C sorce, to a generated big array of chars and offests into that array
one could completely pre-build the hash tables and link it
oh, and also
copying a hash is memcyp
memcpy()
which could be interesting in MVMHash.c 05:01
(assuming that all the entries are "dumb" and don't need pointers fixing or similar)
timotimo hrmpf. 05:19
i think i might have found the the right spot to plop a fact propagation to make read-u?int\d+ methods of Buf jitted more often 05:24
nwc10 cool. (I hope) 05:30
timotimo found another little nice thing to do 05:32
new-disp gave callsites a flag "has a literal in this argument slot" 05:41
some parts of spesh got confused by that and thought the expected types weren't matching, thinking "literal int" being a type that's neither obj/str/num nor int 05:42
stumbled over this while looking around for other places [read|write]u?int\d+ were present 05:43
this would have given us a little performance regression in compilation (the parts that write the mbc, specifically) when comparing master and new-disp
just looking over the post-spesh of write_frame from the MAST compiler 05:48
06:01 Altai-man_ joined 06:04 sena_kun left
timotimo who wants to optimize our truncation and extension stuff :) 06:07
nwc10 not me. already knee deep in hashes 06:09
timotimo grab $!bytecode_offset with sp_p6oget_i32, box it, decont with sp_p6oget_i64, trunc_u32 it, extend_u32 it (i.e. make it 64bit again) and then use it in writeuint as offset
nwc10 that all sounds a bit LTA 06:10
timotimo :D
now these operations are relatively cheap (except boxing, which pushes the nursery allocation pointer ahead just a little) 06:11
but if more or less none of them had to be done ... that would be nicer :)
spesh logs really are huge, huh ... 06:17
my latest changes may also be a miniscule speedup on master 06:19
switching between new-disp and master takes a bit, though, so too lazy to try
is it finally time to write that messagepack format for spesh logs 06:20
06:26 MasterDuke joined
timotimo but a concatenation of messagepack "documents" isn't really a format you can skip through very quickly 06:27
i guess the structure of heap snapshot format 3 could be reused; it has these nice little table-of-content bits that let you skip backwards from the end of the file 06:29
the TOC is relatively small, so you can even spit out the entirety of your outermost TOC whenever you're done writing a batch of stuff 06:30
so at any point (when it's not currently writing to the file, that is) you're free to disrupt the process and the file will still be readable
if the content inside the blobs is zstd-compressed, that can also allow reading from the start, since it has its own way to signal "end of data" 06:31
also, if you zstd compress it, you don't even need to think about splitting strings out of the data body 06:33
like, replacing opcode names with their numbers 06:34
(if you do that, you better put a full table of all opcodes somewhere)
Geth MoarVM: efa8ac2fd4 | (Joshua Yeshouroun)++ | src/core/interp.c
Revert "Declare loop var before loop"

This reverts commit fc092556b9c5a16970b9f92e25e87dc3e3027642.
Now that we are enforcing `-std=gnu99` (23dfde9) this is no longer required.		 06:36
MoarVM: 82ebd0ed69 | niner++ (committed using GitHub Web editor) | src/core/interp.c
Merge pull request #1324 from 0racle/master

Revert "Declare loop var before loop"
MasterDuke .tell lembark if you have a relatively recent Rakudo, there's `VM.request-garbage-collection` (which just calls nqp::force_gc. also, if you're using heaptrack to look for leaks, be sure to run rakudo with the `--full-cleanup` flag. otherwise moarvm deliberately doesn't free some stuff when it exits and it throws off the heaptrack numbers 06:43
tellable6 MasterDuke, I'll pass your message to lembark
timotimo i do believe that's what lembark's already using 06:50
and the program is reaching a very big RSS is the problem as i understand it 06:51
MasterDuke already using what? 07:11
timotimo request-garbage-collection 07:12
MasterDuke yeah, he's trying to find the leak. --full-cleanup was really the only important part of that message 07:13
i looked at his github page to try and find some code to repro with, but i didn't see anything that looked relevant 07:14
timotimo urgggghhh i really don't actually feel like building a version of MVM_spesh_dump that spits out msgpack
MasterDuke nice blog post btw 07:16
timotimo thanks :) 07:17
Geth MoarVM/new-disp: 5 commits pushed by (Timo Paulssen)++ 07:25
timotimo to make the read/write uint ops jitted again also needs an optimization that handles coerce_ui and coerce_iu, at least on known values 07:27
i *think* these are actually basically no-op, but i wouldn't dare just turn them into "set" in production when it turns out there's a condition under which it's not literally just assignment 07:28
BBL
07:53 zakharyas joined 08:02 sena_kun joined 08:04 Altai-man_ left
nwc10 jnthn: but following on from comment previously, even a "1 block" memory layout does not protect against data races during insertion or deletion 08:38
jnthn: although one neat trick I think might help is to keep the hash-block around, but set a bit in it for "this is garbage" and then have the hash lookup code check for that bit, and if set, "oops" the VM 08:49
ie "die early die often" with a backtrace
(at some level)
MasterDuke nwc10: the idea isn't to prevent data races, but just not segfault 09:23
nwc10 the data races will cause SEGVs 09:24
it's data races at the level of C pointer s
MasterDuke timotimo: does github.com/MoarVM/MoarVM/commit/b5...57258113c9 depend on something in new-disp? 09:25
ah. well, i think the idea in general is that we're not preventing data races at the raku level. i.e., you can modify a hash from two threads and we won't guarantee that it does what you want/think, but at least it won't segfault 09:27
nwc10 yes, that guarantee is sane 09:33
jnthn nwc10: What are the pointers involved here? Are there interior pointers within the memory block? 09:34
nwc10 but we can't implement that guarnantee with (only) adding "and save the block of memory and only free it at a safepoint"
jnthn Which get stored?
nwc10 jnthn: well for a regular MVMHash it's a pointer to the MVMString*key and the MVMString* value
but, you only know if the memory locations where those pointers (would be) stored are valid to read 09:35
if the metadata byte for them is set correctly
and the metadata byte gets updated in a different write from the actual hash entry
and during insert and delete, metadata is getting bulk updated 09:36
before the entries get moved around (in bulk)
you can't not do this
jnthn OK, but if the memory was zeroed, then you know you're reading a pointer to an MVMString* or MVMObject* and those are both GC-managed 09:38
nwc10 I think that this discussion is sort of going sideways 09:39
jnthn e.g. So you actually can read them and follow them, even if they were "deleted", safely, at least until the next GC run.
nwc10 it will be easier to see what is possible with code
I think we're talking cross purposes
jnthn Perhaps so. Will have to take a closer look.
nwc10 look for the memmove() calls 09:40
jnthn fwiw, I don't actually care about the safety for anything except the MVMHash case, because all the other uses are VM internal, and we can protect those ourselves.
nwc10 but if we have fast (enough) atomic compare-and-swap 09:41
argh
no, that won't work
I'll shut up and make the current lan work
jnthn :) 09:42
nwc10 "the perfect is the enemy of the good" 09:43
and (it's not clear here) the test version of this is named "MVP" for "Minimum Viable Product"
because that *is* the plan :-)
09:50 MasterDuke left 10:01 Altai-man_ joined
nwc10 releasable6: help 10:02
releasable6 nwc10, status | status link # See wiki for more examples: github.com/Raku/whateverable/wiki/Releasable
nwc10 releasable6: status
releasable6 nwc10, Next release in ā‰ˆ15 days and ā‰ˆ8 hours. There are no known blockers. Changelog for this release was not started yet
nwc10, Details: gist.github.com/f468164059e7076234...1060b2cd8b 10:03
nwc10 and what is tracking the /win lottery these days?
/win a brand new dustbin
/win 6 10:04
oh. nothing :-(
10:04 sena_kun left 10:35 lembark left 10:50 MasterDuke joined 11:08 zakharyas left
nine timotimo: coerce_iu and coerce_ui already share the same JIT implementation and it's exactly the same code as set. 11:56
12:02 sena_kun joined 12:04 Altai-man_ left 14:02 Altai-man_ joined 14:04 sena_kun left 14:38 zakharyas joined 16:03 sena_kun joined 16:05 Altai-man_ left 16:55 AlexDaniel joined, AlexDaniel left, AlexDaniel joined
samcv nwc10, what part of ucd2c.pl do you want to change? Also I'd recommend not rewriting it completely and only iteratively 17:07
nwc10 samcv: *I* don't. (I fixed the few small bits that were really not working 17:22
lizmat was teasing me, or whateer, into doing just what you are suggesting 17:23
but I was rather busy trying to do hashes
lizmat I suggested a Raku version, is what I suggested ?
nwc10 inded 17:24
indeed
and I think that it's the right thing to do
but I'd happily like it to be SEP
lizmat SEP? 17:30
[Coke] someone else's problem
lizmat ah, duh -) 17:36
18:02 Altai-man_ joined 18:04 sena_kun left 18:14 vrurg_ joined
timotimo nine: wonderful! so then spesh can literally just turn it into set :) :) 19:04
19:21 zakharyas left 19:44 vrurg_ is now known as vrurg
Geth MoarVM/new-disp: 6b768846e7 | (Timo Paulssen)++ | src/spesh/optimize.c
"optimize" coerce_iu and _ui by turning it into a "set"
 19:46
MoarVM/new-disp: 0cb67d7aeb | (Timo Paulssen)++ | src/6model/reprs/P6opaque.c
unbox_i/decont_i on P6opaque may be constant-folded

if the value is known, like when it's an enum or something.
This works only because boxing is create-like, not assign- like which means you can't box a different value into an existing object, because you would get a new, distinct ... (10 more lines)
MoarVM/new-disp: 054974fa99 | (Timo Paulssen)++ | src/6model/reprs/P6opaque.c
add spesh comments for unbox_n and unbox_s being optimized

  (on p6opaque objects)
timotimo if i cherry-pick some of these changes, does that make rebasing annoying for jnthn later on? 19:47
19:51 zakharyas joined
MasterDuke good question. not sure 19:54
20:02 sena_kun joined 20:05 Altai-man_ left
nine timotimo: probably not much 20:19
easiest way is rebase -i and just kick them out 20:20
timotimo ah, of course
nine: what do you think should be the logic for when a trunc is followed by or follows an extend and such 20:24
gist.github.com/timo/d4fb520d1abf2...7e6411e361 - if you want to have a look what i'm talking about 20:28
for example in BB 16 20:29
and r14(10) already comes from a box_i that comes from a p6oget_i32 20:30
teaching the exprjit about these ops may already toss out some of this 20:33
i wonder why B20 and B21 aren't merged; we probably never merge inlined and non-inlined blocks together
that is one huge frame 20:54
Frame size: 6848 bytes (2318 from inlined frames) 20:55
Specialization took 12989us (total 21702us)
JIT was successful and compilation took 8205us
Bytecode size: 26594 byte
MasterDuke damn 20:57
oh, wait. us. ok, not quite as long as i first thought 20:58
timotimo ha
yeah
21:16 leont_ left 21:37 zakharyas left 22:02 Altai-man_ joined 22:04 sena_kun left
Geth MoarVM/master: 4 commits pushed by (Timo Paulssen)++ 22:41
timotimo haven't actually spectested it yet
exclamation 23:15
success! 23:16

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!