github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
samcv jnthn: Buf's have no endianess right? 00:05
well i mean they do on the disk. but for perl6's purpose
endianess should only matter when we write the buf16 to the disk right? 00:07
at least how i think of the Buf's is like cpu registers. they don't specifically have an endianess they are just values 00:08
but i could be wrong
timotimo my take on it is that endianness can only come into it if you're moving existing data between different sizes without interpreting the data as its values 00:23
and we currently only support that kind with nativecast
which kind of implies "do it natively, as a cast. it'll be what would happen natively"
and if you use regular assignment, you'll be "stretching" or "squashing" the data anyway 00:24
though of course we'll have to think about this for the low-level byte access api, too 00:25
timotimo i'm not sure if that helps at all, actually 00:26
hm. but you can also pass a Buf, or even a native int array if i'm not mistaken, via nativecall?
geekosaur I'd be carefl wth "cast" C++ has mltiple kinds fora reason 00:28
timotimo right, reinterpret cast is the one that's most like C's cast, isn't it?
geekosaur reinterpret_cast<> could make you unhappy
no, C's vares 00:29
timotimo oh, right
i was refering mostly to casting pointers
geekosaur which is why C++ distinguishes; C gives yo no choice, it['s whatever the compler has decreed for that particular combination of types
timotimo when you cast an int to a float, you'll get the value
when you read a float's value via an int*, you'll get "surprising" results 00:30
i'll go to bed real soon now 00:33
samcv i'm not sure how i'm going to implement writing a file in a different endianess. hmm. gonna need some changes 01:13
since we have our Buf, and then it's in the wrong endianess. so there will need to be some way for us to write it in a different endianess. ugh 01:14
though i mean when we do "foo".encode('utf16be') it *could* encode it as a 16 bit buffer and none of the numbers will match the unicode codepoints since it's in a different endianess 01:15
i guess that could work
probably the easiest way
MasterDuke ugh, if i have an MVMObject, is there an easy way to get the name of the variable it is associated with (assuming of course that there is a named nqp/perl6 variable)? 01:17
not just the REPR and its debug_name? 01:18
samcv no clue, sadly 01:19
MasterDuke heh, may not be necessary anyway, think i just spotted my bug 01:21
and it's one i've made several times before. QAST::Something(:foo('bar')) instead of QAST::Something.new(:foo('bar')) 01:23
but of course, now on to the next bug...`Unknown QAST node type NQPMu` 01:26
of course, i had a QAST::WVar instead of QAST::WVal 01:31
huh, rakudo built, but getting `P6opaque: get_boxed_ref could not unbox for the representation 'P6bigint' of type Scalar` when trying to optimize a range with Int variable end points 01:42
e.g., 'my Int $a = 62; my Int $b = $a + 3; .say for $a..$b;'
the --target=optimize output looks reasonable
MasterDuke oh, think i need an nqp::decont 01:49
samcv those are always annoying to track down. those errors i mean. depending 01:51
[Coke] ff 01:53
samcv i have this weird bug. where somehow decoding a buffer as utf16be causes a segfault 01:54
but it shouldn't. i mean. all it does it alter the order the bytes are read. and utf16be decodes fine in other cases
and it crashes in MVM_string_check_arg 01:55
which is called by MVM_string_concatenate
which makes NO sense at all... given there is almost no difference in how it processes utf16be and utf16le
well if i make the utf16le function actually call the utf16be function and try and .decode('utf16le') (which will actually call the big endian code) 01:58
then everything works fine and no crash... ugh
oh. looks like i didn't predeclare the function. we need to fix those debugserver.c warning messages. i always miss my own warnings since i often just start ignoring all warnings when there's so many 02:00
MasterDuke there's some weirdness with gcc versions and those warnings. i commented on the most recent commit that silenced them for some combinations of compiler/platform 02:02
samcv: github.com/MoarVM/MoarVM/commit/62...098e5f3ab3 02:03
samcv we need to use PRIi64 or PRIi32 etc always. unless our actual variable is an int, or long int and not sized
MasterDuke: clang complains 02:04
at least 02:05
about how it is atm
clang 6
samcv hmm i got an asan use after free heap error running nqp's test suite 02:51
but it only happened once
gist.github.com/927ebf8b4a1c3ae711...7d9da00d34 does not seem anywhere at all related to what i changed
Geth MoarVM: b162c7c4f5 | (Samantha McVey)++ | 5 files
Add utf16le and utf16be encoding types and impl. for buf decode

We add utf16 little endian and utf16 big endian as encoding types. For now it is only implemented for buffer decode.
At the moment these new encodings will ignore and pass through any byte order marker (BOM) as stated in RFC 2781. ... (10 more lines)		 02:52
MoarVM: 0dc1e96e23 | (Samantha McVey)++ | src/strings/ops.c
Fix last commit so it is C90 compatible and gcc doesn't complain

Make sure declarations are before code.		 04:09
MoarVM: adce51fb80 | (Samantha McVey)++ | src/debug/debugserver.c
Fix printf compiler warnings of mismatching type

Use the system agnostic PRIu64/PRIu32 instead of %u and %llu/%lu. This way we avoid it giving warnings on one persons computer while the other may not give warnings due to the compiler setups.		 04:20
travis-ci MoarVM build failed. Samantha McVey 'Fix last commit so it is C90 compatible and gcc doesn't complain 04:28
travis-ci.org/MoarVM/MoarVM/builds/429404682 github.com/MoarVM/MoarVM/compare/b...c1e96e23c8
travis-ci MoarVM build passed. Samantha McVey 'Fix printf compiler warnings of mismatching type 04:40
travis-ci.org/MoarVM/MoarVM/builds/429406030 github.com/MoarVM/MoarVM/compare/0...ce51fb805e
Geth MoarVM/fork-safety: fb9248b4dd | (Bart Wiegmans)++ | src/io/procops.c
[fork] Reinitialize libuv in child, move locking a bit

libuv locks are not guaranteed to be recursive (and indeed they aren't on linux), so we need to unlock the mutex_event_loop prior to restarting the io thread.
  ugexe++ mentioned that we need to call uv_loop_fork to reinitialize
event loops in the child process.		 05:56
Geth MoarVM/speshplugin_guardstaticcode: 14d8fa6edc | (Timo Paulssen)++ (committed by Bart Wiegmans) | 8 files
add speshguardgetstaticcode, for closures and such

lets a spesh plugin figure out if a given object, such as the $!do attribute of a Code, is "the same" across invocations - ignoring what exactly it closes over.		 06:40
brrt ^ rebase
brrt \o 09:15
nwc10 o/ 09:19
jnthn \o 09:20
brrt wonders about the practicality of removing the per-threadcontext uv loop 11:07
I think we only use it for directory operations
and stat and things like that
lizmat brrt: so you're suggesting they should be started on-demand, or be shared by a single thread? 11:27
if the latter, wouldn't that have potential issues with results bleeding through different threads ? 11:28
brrt hmm. No 11:37
what I am suggesting, is that we do not maintain a uv_loop structure per thread, at all 11:38
all actual async IO is handled by the eventloop thread
I wonder if it may be worth it to just start that eventloop thread anyways
so for context - currently we maintain a uv_loop_t per MVMThreadContext object 11:39
previously, all async IO would have gone over those, but indeed that lead to problems with sharing between threads
that is why (iirc) the eventloop thread was introduced 11:40
so that all async IO operations go over that thread, they aren't shared between multiple uv_loop_t's,
lizmat wonders what jnthn thinks about it :-) 11:41
brrt yeah, me too
I think we don't need them anymore. But I'm not sure
timotimo o/ 11:56
brrt ohai timotimo 12:00
I rebased a branch of yours
do you want that branch merged?
timotimo i built it for 5b881e9f2 12:02
last i tried it didn't actually work, though?
brrt hmm 12:30
well, maybe let's not merge it just yet, then
timotimo there's no harm in making it available
just the spesh plugin isn't correct yet
lizmat perhaps wait until after the 2018.09 release ? 12:31
jnthn It may be that we can replace various of the things we use the per-thread libuv event loop for with a small bit of platform-specific logic for POSIX and Windows. 12:32
yoleaux 11:23Z <Zoffix> jnthn: FWIW there are 4 @LARRY 6.d Issues that could use a comment: github.com/rakudo/rakudo/issues?q=...3A%40LARRY (also there's a question for you on github.com/rakudo/rakudo/issues/2286 about $!reified)
timotimo the more libuv we kick out, the closer we get to running moarvm on platforms that don't have a libuv port ... 12:33
jnthn Most do by now though
And we still need it for async I/O and that lot we're *not* going to reinvent :)
timotimo right 12:34
definitely
brrt jnthn: that was my thought as well 12:59
most remaining usage I see, is actually synchronous operations
(it's interesting how local file IO doesn't have async versions) 13:00
diakopter brrt: I thought libuv had async local file IO; moarvm just didn't use them 13:12
diakopter at least, I have a very vague memory of asking jnthn "hey should we expose libuv's async file IO yet" with a "nah" reply :D 13:16
brrt I seem to recall that linux doesn't have it 13:18
so whatever libuv has, must be limited :-)
diakopter well it can be emulated with threads, I thought 13:20
jnthn And if it can be emulated with threads we can do that at Perl 6 level :) 13:25
timotimo so worst case we're attempting to access like 10 files on a very very slow nfs or sshfs and we have to have 10 threads so that we react to the first one that replies? :P 14:25
brrt "don't do that, then" :-P 15:04
diakopter timotimo: how well is the Man_Or_Boy benchmark test JITted these days? :) 15:09
Geth MoarVM: ddde095083 | (Samantha McVey)++ | 3 files
Add encode support for utf16le and utf16be

  * These don't place any byte order mark (BOM)
  * These are not planned to ever place a BOM by default though we may
   have a configurable option		 15:10
diakopter hunh; where'd the manorboy.nqp go 15:12
diakopter wasn't it once under nqp/t/benchmarks or something 15:14
diakopter nqp: my $in := 10; sub A ($k, $x1, $x2, $x3, $x4, $x5) { my $B; $k <= 0 ?? $x4() + $x5() !! ($B := sub () { A(--$k, $B, $x1, $x2, $x3, $x4) })(); }; say(A($in, my sub(){1}, my sub(){-1}, my sub(){-1}, my sub(){1}, my sub(){0})) 15:17
camelia -67
Geth MoarVM/postrelease-opts: 65 commits pushed by (Jonathan Worthington)++, (Timo Paulssen)++, MasterDuke17++
review: github.com/MoarVM/MoarVM/compare/0...2d3bd72158		 15:53
jnthn Just a rebase :)
Seems that it retains its various wins, though :) 15:55
timotimo diakopter: no idea; it looks a little like it does deep recursion? 15:59
diakopter aw, profiling it at k=20 causes segfault
timotimo k is $in? 16:00
diakopter yeah
timotimo perhaps it's to do with memory exhaustion?
diakopter maybe I suppose. it doesn't come close to exhausting memory when not profiling 16:01
nqp: my $in := 10; sub A ($k, $x1, $x2, $x3, $x4, $x5) { my $B; $k <= 0 ?? $x4() + $x5() !! ($B := sub () { A(--$k, $B, $x1, $x2, $x3, $x4) })(); }; sub mob($in) { A($in, my sub(){1}, my sub(){-1}, my sub(){-1}, my sub(){1}, my sub(){0}) }; mob(10); say(mob(20)); 16:05
# attempt at warming up the jit
camelia -175416
diakopter aww, running the profile for k=14 gives "nqp --profile mob.nqp 16:10
-175416
oops
Maximum call stack size exceeded in JSON.parse (in Chrome console) 16:11
sigh 16:13
so it embeds a copy of the call tree 16:16
JSON could handle it if it was flattened 16:17
diakopter how many invocations are needed to trigger the JIT 16:19
apparently it's not counting recursive invocations as invocations
alright if I warm up the JIT by calling mob(2) 2000 times, mob(20) takes ..... longer 16:21
EJITSLOWER 16:22
(still can't profile above k=11) 16:26
timotimo did you have a look at the spesh log and/or jit log? 16:44
diakopter in the profile output? 16:45
timotimo nah, the one you get when you set MVM_SPESH_LOG or MVM_JIT_LOG to apath 16:46
diakopter I can see the jit invocation stats with k=11, but can't really quantify the improvement when I can't profile above k=11
timotimo you can time the whole execution with MVM_JIT_DISABLE on or off 16:52
nine Huh? How would I get at the single bytes in a num64? Or otherwise write it to an uint8 array? 19:02
timotimo nine: only way i know is nativecast or CUnion 19:03
jnthn nine: By implementing the new ops I proposed that do exactly that! :) 19:04
nine nqp::nativecallcast only takes pointers
timotimo yeah 19:05
you need to put them into bufs
or CArray
also, what jnthn just said
nine I guess since I've already had to add that buffertocu op, I'm off the "keep the changes purely in NQP" path already anyway... 19:07
timotimo :) 19:08
brrt` go for it, I say 19:10
lizmat and another Perl 6 Weekly hits the Net: p6weekly.wordpress.com/2018/09/17/...ersus-six/ 21:47
jnthn lizmat++ 21:55
Geth MoarVM: 792cdd5807 | (Samantha McVey)++ | 2 files
Add support for utf16le and utf16be decodestream

There are some issues with takechars, but .lines and .slurp works. It omits the byte order mark if and only if it is at position 0 in the file.		 23:30
MoarVM: ad12d8e4d7 | (Samantha McVey)++ | 2 files
Fix some bugs with utf16* decodestream get_chars, and retain state

Retain state i.e. which encoding we are currently using. This is important for utf16 encoding since it detects using a byte order mark whether it is big or little endian. We must save this detection so further calls use the proper encoding.
Also fix issues with get_chars, which the location could get off if you were reading one unicode codepoint at a time instead of slurping everything.

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!