github.com/moarvm/moarvm | IRC logs at irclog.perlgeek.de/moarvm/today
Set by moderator on 5 May 2018.
MasterDuke timotimo: wasn't there a commit somewhere recently about including the time the jit takes to compile stuff in the "dynamic optimization" field of a profile? 01:06
timotimo no, it only shows up in the spesh log 01:07
it's already in the "dynamic optimization" section of the profiler :)
just not split in parts
MasterDuke ah, ok, thanks
timotimo JIT was successful and compilation took 2204us 01:08
^- lines in the speshlog look like this
MasterDuke "call frame" in the profile, is that pretty much equal to a block? 01:10
or more to a routine? 01:11
timotimo we try to inline blocks into their containing blocks in the static optimizer, so there isn't a 1:1 correspondence, but other than that, block is the better representation 01:12
we also have thunky things that aren't blocks but get their own call frames, or at least i think they do
like whatevercodes, "where" clauses, default values for parameters or attributes, LHS of the xx op, ... 01:13
MasterDuke hm. i'm trying to add some simple information about how to interpret profiles to the docs 01:14
i think i'll add an `(i.e., blocks)` after mentioning call frames, seems close enough 01:15
timotimo i'll head to bed before i make any further comments :P 01:20
but that does sound good as a first approximation
MasterDuke yeah, i'm just planning to get some minimal explanation in there, people who know more can fix and/or flesh it out 01:21
01:25 colomon_ joined 01:57 ilbot3 joined
moderator github.com/moarvm/moarvm | IRC logs at irclog.perlgeek.de/moarvm/today
04:46 nebuchadnezzar joined 04:49 nwc10 joined 07:25 robertle joined
Geth MoarVM: 6899b0a172 | (Stefan Seifert)++ | 2 files
Serialize hashes sorted for more reproducible builds
 08:14
nine I don't like the code in ^^^ very much, but I also don't have a good idea on how to improve it.
yoleaux 5 May 2018 13:25Z <MasterDuke> nine: your make_builds_reproducible_again branches seem to have fixed it (assuming i tested correctly)
5 May 2018 13:26Z <MasterDuke> nine: i pulled latest rakudo, reverted a commit, `git reset --hard`, tried to rebuild and got errors in make test and install. then i rebuilt with nqp and moar on those branches, and tests passed and install succeeded
08:41 domidumont joined 08:48 domidumont joined 09:17 brrt joined
brrt .tell nine you probably want to use qsort_r on gnu, and qsort_s on windows 09:22
yoleaux brrt: I'll pass your message to nine.
09:39 yoleaux joined
nine brrt: yes, I'd want qsort_r on GNU. Didn't know about qsort_s though. But what about other platforms? 09:48
yoleaux 09:22Z <brrt> nine: you probably want to use qsort_r on gnu, and qsort_s on windows
brrt are there any others :-p 10:04
lemme check bsd
www.freebsd.org/cgi/man.cgi?query=...ease-ports freebsd has qsort_r 10:05
the other option is to just write a merge sort / quicksort ourselves 10:07
nine For qsort_r we'd have to define _GNU_SOURCE on Linux
brrt yeah 10:08
hmm
none of those options is really attractive 10:09
but i'm pretty sure having a variable for the tc, no matter how temporary, is a recipe for failure
especially at the margin
because MVM_string_compare can in fact throw
and what if two threads happen to be sorting at the same time 10:10
nine I figure, two threads writing byte code at the same time is....at least quite far off 10:11
brrt hmm 10:14
true
:-)
anyway, maybe the solution to this and other sorting woes, is just to have our own wrapper 10:15
nine I'd have to find the energy to ever finish the precomp-singleprocess branch. And then find a way to parallelize the builds ;)
OTOH maybe I have gathered enough knowledge by now to actually give this a try again 10:18
I guess segfaults in spectests are bad. But what about segfaults in the test harness? Specifically the perl 5 harness?! 10:40
brrt segfaults are bad 10:50
but they have usually one redeeming quality
they (tend) to happen at fixed points
yay, i can actually assemble and link the stack walker for w64 10:52
Geth MoarVM/jit-stack-walker: e1e7377feb | (Bart Wiegmans)++ | 4 files
[JIT] Implement stack walker for windows

Same as for POSIX, really, but MASM just spells things very differently, and seems to think that there are such things as
  'blocks' in assembly. I've decided to give them different suffixes
  (.asm on windows, .s on posix), and to use preprocessor macros to
pick the right ones.		 10:54
brrt the only remaining hurdle is to configure the assembler correctly per toolchain 10:55
nine Ok, a reboot seems to have gotten rid of the segfaults. I guess I'm running my memory at the very limit the board can drive it stably 11:04
brrt for some reason i can't seem to get magit installed on my vm 11:14
highly distressing if only because it shows how much you can get accustomed to your tools 11:15
it also turns out that if i don't specify the assembler by force, both msvc and gnu/clang know just what to do 11:17
so i kind of feel like leaving it as is
segv :-( 11:23
i wonder what i'm doing wrong on windows 11:27
jnthn Actually, we *can* be assembling bytecode on two threads already, if the EVAL survives that far :) 11:40
yoleaux 09:02Z <nine> jnthn: In NQP classes keep track of the order of their methods but roles do not. I find this odd. Does method order matter or not?
jnthn (Which it can, 'cus I've done it, though it's not the most stable thing...)
brrt it seems my stack is corrupted on windows ... 11:41
jnthn .tell nine Yes, method order matters for token/rule/regex since source order is an LTM tie-break
yoleaux jnthn: I'll pass your message to nine.
nine jnthn: OK. Anyway I've already pushed commits to nqp which retain the order of methods in roles. 11:42
yoleaux 11:41Z <jnthn> nine: Yes, method order matters for token/rule/regex since source order is an LTM tie-break
jnthn Nice, maybe you fixed a bonus bug along the way then ;) 11:45
Hm, that tc hack is horrible :P
brrt i kind of want to avoid having our own reentrant qsort implementation 11:50
*ahem*, -fno-omit-frame-pointer is spelled '/Oy-' in windows land
nine samcv seems to have managed that platform dependency mess nicely with get_random() and friends
brrt not '/Oy'
jnthn It's short for, "Oh, you ------" 11:51
brrt otoh, the msys version does have the -fno-omit-frame-pointer 11:53
and
it still doesn't appear to work
jnthn Perhaps we can have our own reentrant merge sort implementation instead, to avoid accidentally quadratic :P 11:55
nine If your code gets large enough that sorting algorithms in the compiler start to matter, you have much larger problems already ;) 11:57
OTOH Java's developers probably thought that, too and now we're running into arbitrary limitations with rakudo-j :D 11:58
jnthn 16-bit numbers should be enough for anyone :P
I was surprised how many things in a class file used those :) 11:59
Though...this is what, 25 years old design decisions, and with a class file per language-level class, it is still pretty reasonable today, provided the source language is JavaScript :) 12:00
oops
*Java
Geth MoarVM/jit-stack-walker: 11 commits pushed by (Bart Wiegmans)++
review: github.com/MoarVM/MoarVM/compare/e...a1e14709f6		 12:01
brrt rebase + assembler rule 12:02
12:02 squashable6 joined
brrt i'm totally confused why this doesn't just work 12:02
but i'll have to get out the debugger big guns at some other point
12:04 squashable6 joined 12:06 squashable6 joined
brrt the thing is, if i step through it with gdb, it works for the first 5 or so frames, then failes 12:13
wtf
maybe ldflags also needs a -fno-omit-frame-pointer? 12:15
oh hey... msdn.microsoft.com/en-us/library/w...s.85).aspx 12:20
MasterDuke hm, has anyone tried running rakudo on an Itanium? 12:21
13:51 AlexDaniel joined
timotimo src/platform/memmem.h:19:12: warning: implicit declaration of function ‘memmem’; did you mean ‘memset’? [-Wimplicit-function-declaration] 15:12
?!?
ok, had to put the include before #include "moar.h"
m( i'm getting multiple definitions of MVM_memmem 15:14
oh, it was right all along, i just had to ignore the warning that memmem was being used but not defined 15:16
OK, here's the correction: the piece of code we're interested in does appear in the serialized blob. at offset 5e18
15:18 domidumont joined
timotimo found the thing (variant B) in section objects data at offset 26a8 15:24
it's getting more and more interesting
15:36 SmokeMachine joined
samcv good * everyone :) here with lizmat 15:52
yoleaux 11:12Z <AlexDaniel> samcv: please let me know if you need a ticket this: irclog.perlgeek.de/perl6/2018-05-06#i_16134827
timotimo yo samcv
lizmat o/ 15:59
Geth MoarVM: e799a6c278 | (Timo Paulssen)++ | src/6model/reprs/P6opaque.c
improve debug helper for p6opaque dumping

used to explode violently when an inlined attribute was encountered that couldn't unbox to a string.
Also segfaulted when encountering a type object.		 16:56
MoarVM: cb3ac08dbe | (Timo Paulssen)++ | src/6model/reprs/P6opaque.c
fix warning about missing cast
 17:08
17:13 lizmat joined
Geth MoarVM: d5f57dcc67 | (Samantha McVey)++ | src/strings/ops.c
In str index avoid checking for concreteness multiple times

Get the number of graphemes after checking the string args so we don't have to perform the checks multiple times.		 18:19
MoarVM: 0c793f88ca | (Samantha McVey)++ | src/strings/iter.h
Rename ci->base_code to ci->first_code to avoid confusion

We used to assume that the first codepoint of a synthetic was the base codepoint. Back then there were no "Prepend" Unicode codepoints so this assumption held. For a while we have supported Prepend codepoints, so that is fine. Rename the variable so its name is accurate to its function.
18:23 brrt_ joined
brrt_ good * 18:23
brrt` i wonder where i'm still logged in? 18:24
anyway
i think i figured out a likely culprit for the crashes on windows 18:25
timotimo is it windows?
brrt` i think it is my use of alloca
no
i managed to crash windows, though
but that may have been due to vm usage 18:26
18:28 brrt joined
brrt heh, it was the same emacs session that had an irc open 18:29
samcv good * 19:17
brrt: how much did you allocate?
brrt tiny bits 19:26
but the point is, it was happening on a stack that needed to be walked 19:27
and i have reason to believe that on windows, using alloca somehow offset the frame address
the reason i believe that is that i saw my stack walker derail in a frame that did an alloca 19:28
samcv hmm 19:29
brrt not a problem usually though 19:30
samcv is this windows doing something against C standard, or are you depending on unspecified sane-ness 19:31
brrt i'm not entirely sure 19:48
the C standard specifies nothing about the structure of the stack 19:49
then again, the ABI does, kind-of
or rather
the convention is that the frame pointer (rbp) is properly saved and restored in every frame, just above the return-address pointer
the return address pointer is always supposed to be there, otherwise the 'ret' instruction would never work 19:50
this convention has recently been broken when -omit-frame-pointer became the default, but having it is still the right way 19:51
19:58 Kaiepi joined
brrt lets see if my fixes work 20:04
Geth MoarVM/jit-stack-walker: e893a297fa | (Bart Wiegmans)++ | 4 files
[JIT] Use iterators rather than array output

Array output necessitates allocating a buffer. It is my belief that using alloca() can have the side-effect of modyfing the frame pointer on windows (because during debugging the thread cursor derailed after a frame using alloca()), so lets try the same without.		 20:15
brrt hmm, spectest isn't entirely clean so far 20:19
MasterDuke brrt: fyi, your `i < jitcode->num_handlers;` line and the next are indented oddly in a bunch of places 20:24
brrt oh, emacs isn't respecting my tabs? 20:26
MasterDuke: sorry, which lines? shows up normal for me 20:27
nb: github sometimes collapes whitespace preceeding long lines
MasterDuke exceptions.c lines 109 and 110 look like they're indented one more space than 111 20:30
brrt that's intentional, they're within the parentheses of the for loop
anyway, see you tomorrow 20:31
MasterDuke oh, of course. guess i need an afternoon coffee to look any other code... 20:55
21:19 quotable6 joined 22:00 ilbot3 joined
moderator github.com/moarvm/moarvm | IRC logs at irclog.perlgeek.de/moarvm/today
timotimo haha 22:36
oh that's fun
so, you know how ops to get optional parameters have a label as third argument
they jump to that label if the optional parameter exists 22:37
turns out the allocation logging put the allocation log instruction right after the op instead
so if something got created, like with autoboxing, it won't get logged because the instruction is jumped over
if nothing got created, that gets fed to the logger, which will see it's not worth logging and shrug it off
that's what caused liz' recent code with a for loop that only does Nil to do GCs but apparently not allocate according to the profiler
one bug down! 22:42
japhb
.oO( Black Bug Down )
 22:43
Geth MoarVM: 8521118b17 | (Timo Paulssen)++ | src/profiler/instrument.c
profiler: fix optional parameter allocations

these ops jump to a label if the parameter was present, but the allocation logging op was inserted right after the op. That meant that whenever something was allocated by these ops, i.e. due to autoboxing, the logging was skipped over.
Instead of that, the logging ops now go in the right place.		 22:45
MasterDuke how many allocations was that missing? 22:46
timotimo every time an optional object parameter got a native value passed at the very least
the code in question, for ^100_000 { Nil }, ends up binding the $_ to a lexical, which is why it doesn't just get thrown away 22:47
MasterDuke heh, now i get 99985 Int allocations 22:51
timotimo++ 22:52
timotimo not entirely sure where the extra few went
but we don't promise exactness anyway :P 22:53
22:53 committable6 joined, unicodable6 joined
timotimo that does look weird 22:54
22:54 undersightable6 joined
MasterDuke well, it's a lot better than the 4 lizmat said she got before 22:55
timotimo yeah, but the spesh graph still looks a bit strange 22:57
the allocation logging instruction seems to land in the wrong BB again??
MasterDuke also, the overview says 0 GCs were performed, but the GC tab shows info for 1 22:58
23:43 yoleaux joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!