github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
MasterDuke timotimo: you around? 01:53
timotimo i am now 02:06
MasterDuke: what's up? :)
MasterDuke does www.joyfulbikeshedding.com/blog/20...k-trimming look relevant? 02:07
timotimo oh! i like that visualization! 02:09
MasterDuke would it make sense to add a malloc_trim(<something>) to full GC runs? 02:16
timotimo we could surely try
do we really only have to call malloc_trim(0) or so?
seems so 02:17
wanna give it a try? 02:19
MasterDuke in here? github.com/MoarVM/MoarVM/blob/mast...#L226-L232 02:20
huh. seems to cause a `Command terminated by signal 11` when profiling 02:23
timotimo that'd probably be a fine spot
ouchies
MasterDuke but does reduce maxresident for 'for ^50 { my $cmd = run « cat input.txt », :out; $cmd.out.close; }' from ~1132152maxresident to ~878972maxresident (only measured on a couple runs) 02:24
timotimo that's certainly noticeable
MasterDuke where input.txt is 130M and 2306812 lines
a little slower though. 0:08.19elapsed to 0:08.47elapsed 02:25
any idea how to fix profiling? 02:28
the "command terminated..." happens after the "Writing profiler output to..." 02:29
timotimo well, without knowing what caused the segfault, dunno 02:35
MasterDuke (gdb) bt #0 0x0000555500000002 in ?? () #1 0x00007ffff79a9ecd in MVM_coerce_smart_stringify (tc=0x55555555acb0, obj=<optimized out>, res_reg=0x555557e3eff8) at src/core/coerce.c:289 02:39
with MVM_SPESH_DISABLE=1 02:40
timotimo oh? i'd say "tell me what the object is", but it's probably b0rked all to hell 02:43
MasterDuke turned off optimization, now have `$1 = (MVMObject *) 0x55555866df70` 02:46
(gdb) p REPR(obj)->name 02:49
$4 = 0x0
timotimo what address does the REPR have? 02:53
perhaps the object itself is already bogus
MasterDuke oh, it works fine under valgrind
timotimo turning the GC debug thing on could perhaps give an earlier detection of the problem
haha, of course it would
could be a multithreading-exacerbated problem 02:54
MasterDuke should that malloc_trim() happen in the coordinator instead of each thread? 02:55
here perhaps? github.com/MoarVM/MoarVM/blob/mast...ate.c#L187 02:56
nope, exact same behavior 02:58
well, i have to afk for tonight. might not be available much the next couple days, hopefully you can figure something out 02:59
samcv morgen 06:56
brrt morgen samcv 07:44
timotimo so, how do we feel about zstd in moarvm? should i try to do it all via dlopen, or would it be fine to statically or dynamically link an actual zstd library? including it in 3rdparty yea/nay? 14:16
building a derived zstd that only encodes and only contains a single compression level, would that be a thing? 14:18
jnthn Hmm, issues are 1) size, 2) licensing, 3) what packagers would make of it :) 14:21
(Debian doens't like embedded things, for example)
timotimo This dumper is also not thread-safe. It does not acquire any ptmalloc2 mutexes. When using it, make sure that the process you're dumping is idling in all threads. 14:22
Although I haven't explicitly tested, this dumper is probably incompatible with swapping. It uses mincore() to check whether a page is in use or whether it has been madvise(MADV_DONTNEED)'ed, but I suspect that if a page has been swapped to disk then mincore() would return the same result. So disable swap space.
^- that's the malloc heap dumper/visualizer thing from that article about ruby
not exactly fantastic, but if i put it in the right spot in the gc it'll probably work
zstd is dual-licensed BSD and GPL2
moar should probably not fail to build if zstd is missing, it'd just mean that it wouldn't write heap snapshots, or if it does it wouldn't compress them 14:25
jnthn I think "don't compress" is the more resonable fallback.
timotimo given the compressed size is about 5% of the uncompressed size ... 14:26
jnthn Hm, what happens on Windows? MacOS? 14:27
timotimo yeah, "dump uncompressed" is probably the right fallback after all. or keep the version 2 format around 14:28
jnthn hmm, I just merged postrelease-opts, but it doesn't build 14:55
src/jit/x64/emit.dasc: In function ‘MVM_jit_emit_primitive’: 14:56
src/jit/x64/emit.dasc:796:5: error: duplicate case value
case MVM_OP_sp_bind_i32: {
^
src/jit/x64/emit.dasc:768:5: error: previously used here
case MVM_OP_sp_bind_i32:
^
timotimo oh, a merge conflict that wasn't properly resolved?
jnthn Wonder if that's merge fail or a problem in the work pushed to that branch...
brrt`` probably yes
timotimo or something cherry-picked?
jnthn Well, there wasn't a conflict there
timotimo well with lines that far away it wouldn't conflict
jnthn Looks like I can remove the second ocurrence since the first is doing the same codegen 14:58
timotimo OK, so i got three different formats for the same heap snapshot now 15:12
one is the current "stable" format, aka v2; the file is 1.1 gig big 15:13
another format is all individual fields in individual text files as ascii numerals, all gzipped individually, which is 161 megs big
the last one uses the framing format that i'll use for the upcoming format. it currently contains gzipped sequences of same-sized raw integers for individual fields; it's 147 megs big 15:14
decompressing the gz files and recompressing them as zstd should give me an idea how big the final new format size will be when using zstd
Geth MoarVM/master: 20 commits pushed by (Jonathan Worthington)++, (Timo Paulssen)++
review: github.com/MoarVM/MoarVM/compare/7...7810ce7ca9
15:31
MoarVM/more-pea: 4 commits pushed by (Jonathan Worthington)++ 15:40
timotimo hm, catting all the zstd files for one number series together gives about 6x as much memory usage than catting the source files and compressing them in one session 15:41
but i'd prefer being able to random access the individual snapshots 15:42
jnthn The moar-pea push was a rebase 15:47
Now trying to reproduce the asan explosion nwc10 mentioned that branch had 15:48
Doesn't seem to happen with normal configuration
nwc10: Can you still reproduce the asan explosion with latest more-pea (rebased on latest master), and if so, what have you got set besides MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 ? 15:51
travis-ci MoarVM build errored. Jonathan Worthington 'Merge branch 'postrelease-opts'' 15:51
travis-ci.org/MoarVM/MoarVM/builds/506829978 github.com/MoarVM/MoarVM/compare/7...7810ce7ca9
jnthn epic fail, that benchmark I thought had got slower hadn't; I'd modified the Perl 6 version of it and then forgotten that 15:55
timotimo whoops. but yeah, i've done that kind of thing before, too
Geth MoarVM: 5e3633987d | (Jonathan Worthington)++ | 2 files
Avoid a bunch of malloc/free churn in code-gen

We can use the same arrays over and over again, so just clear them instead.
18:45
timotimo huh, how does that macro work? 18:47
i mean the init one also uses "x ## _foo"
travis-ci MoarVM build passed. Jonathan Worthington 'Avoid a bunch of malloc/free churn in code-gen 19:06
travis-ci.org/MoarVM/MoarVM/builds/506922411 github.com/MoarVM/MoarVM/compare/6...3633987d12