01:48 ilbot3 joined 05:11 geekosaur joined 05:35 domidumont joined 05:39 domidumont joined 06:02 domidumont joined 08:34 harrow` joined 08:36 psch_ joined
domidumont Weird, arm64 tests pass on a qemu chroot, but hangs on real arm64 HW 08:40
The hangs involve precompilation phase. It hangs only when the precompiled files are missing. See paste.debian.net/864333/ 08:41
When the test hang, the CPU is idle
HTH
brrt good * #moarvm 08:42
that definitely sounds like a deadlock too me 08:43
domidumont: ^
domidumont brrt: I've seen similar symptoms on qemu/arm64 chroot before this fix github.com/MoarVM/MoarVM/commit/17...a347920a64 08:46
brrt I'm puzzled as to why that should make a difference 08:47
domidumont it may be a mix of different issues. We also had test failure on armhf. As far as I know, only arm64 lead to tests hang 08:49
08:49 dalek joined, flaviusb joined, synopsebot6 joined, JimmyZ joined, BinGOs joined
domidumont Hmm, test don't hang with MVM_SPESH_DISABLE=1 08:55
anything else I should try ? 08:56
10:36 FROGGS joined 11:12 cygx joined
cygx o/ 11:12
another fun Unicode issue: github.com/MoarVM/MoarVM/issues/427
jnthn Yup, they went and added prepends...
Only a problem since Unicode 9 11:13
cygx and forgot to give a sensible update for the definition of base char
jnthn Not sure on ord yet
Question is if we want ord($c) and ords($c)[0] to be different 11:14
But as for deriving character properties, it's fairly clear we don't want to base *those* on the prepend
Same for the tc/lc/uc/fc 11:15
cygx keeping .ord as an alias for .ords[0] sounds ok to me (assuming a replacement like .base gets added) 11:17
jnthn .base is taken :)
I could probably go with changing .ord also
cygx .base-char, .unibase, .first-non-prepend-char ;)
jnthn I don't think we promised .ord and .ords[0] equivalence anywhere 11:18
docs.perl6.org/routine/ord#(Str)_routine_ord says "base char" :) 11:20
cygx then, all good ;)
jnthn aye :) 11:21
Talking of good things...lunch :) &
11:31 stmuk_ joined 12:49 zakharyas joined
cygx jnthn: in my long tradition of throwing out half-baked ideas, here's another one on how to deal with synthetics representing denormal and compatibility input: github.com/cygx/p6-newio/blob/master/synth32.h 13:00
13:12 ilmari joined
timotimo i'm watching Nicholas Ormrod's talk on "std::string at facebook" and they (in their fbstring) and clang (in their std::string) both have a representation of strings where short strings can live directly in the space that used to be for pointer, size, and capacity 13:54
we don't actually have a capacity value, so we'd have to make do with only 2/3rds as much space for "small strings"
though perhaps the member that'd usually encode what kind of storage we're using can become a part of this mess
oh, and of course their characters are probably 8bit
ok, i think this approach is not right for us. 13:55
they replaced std::string with fbstring and measured a 1% performance win across all of www.facebook.com 13:57
cygx well, we do special case ASCII strings, and for those, inlining the data into the MVMStringBody might be a win 14:03
depending on architecture and whether we want to cache the hash code or not, there's place for 10+ chars
timotimo we *do* special case them ... but we don't have any code that deals with ascii strings, if i'm not mistaken 14:04
like, i don't think a single 8-bit-per-char string is alive in any moar instance out there
especially since we still have to turn every string into a flat MVMGrapheme32 string in order to calculate hashes 14:05
cygx we do get a fwe special cases in strings/ops.c
*few
timotimo OK
ideally we'd be using the grapheme iterator api for everything
cygx or go the opposite direction: always use 32-bit codepoints, but flag the range (ascii-only, latin-1, 16-bit single-codepoint graphemes, 32-bit single codepoint graphemes) 14:08
[Coke] github.com/perl6/doc/pull/964/files#diff-0 - that's not a twigil.
... wrong window
cygx we could also put those into a flexile array mamber to reduce the number of allocations 14:09
*member
timotimo i'm not sure what you mean by that
you mean "this string never contains anything outside of 0..127" for example? 14:10
cygx yes
could be used for various optimizations
timotimo it'll allow us to fast-fail for same-length strings where one contains silly characters and the other doesn't and we want to eq them 14:11
cygx or encoding ASCII-only to UTF-8
timotimo mhm
cygx a flexible array member might not be a win if we then have to move the string contents around during GC 14:13
timotimo i don't know what "a flexible array member" is?
cygx struct { size_t size; uint32_t codes[]; } 14:15
pre C99, you used zero-length array for that
timotimo oh 14:16
we can't have that kind of thing yet
our GC doesn't understand that
cygx the GC does not support variable-sized objects? 14:17
where does the size live - the STable?
timotimo let me stop speaking about this and have a look before i embarass myself :)
the size actually lives at the end of the header of the object 14:18
jnthn The object's size can't change during its lifetime, but the size is stored in the header. But it's not intended you'll allocate large arrays directly into the nursery.
(The size is actually stored in 16 bits)
timotimo the only examples i know where we can have different sizes for object building is p6opaque and cstruct 14:19
in p6opaque, we allocate st->size bytes for the object
cygx I see
timotimo the thing about putting a string's body (or the same for arrays) inside the object itself is that it's only worth it if it's sufficiently small 14:20
on the other hand, that technique is usually combined with the fact that the objects are often allocated on the stack
cygx what could be tried is this: always have 32-bit units in regular strings but mark the range and embed short latin-1 strings directly 14:21
timotimo so you don't have to go from stack to heap at all, whereas we currently always allocate on the heap
cygx you could reserve a couple of bytes for that purpose at the end of the structure
arnsholt Putting the characters in the body of the string would save you a level of indirection though
cygx you would end up with regular 32-bit strings of RANGE_FULL, RANGE_ASCII, RANGE_LATIN1, RANGLE_BASIC16, RANGE_BASIC32, strands and embedded short latin-1 strings instead of the current variants GRAPHEME_32, GRAPHEME_ASCII, GRAPHEME_8, STRAND 14:24
might be a loss if there are a lot of long-ish
... GRAPHEME_ASCII, GRAPHEME_8 strings
timotimo i had a bit of code lying around for caching strings that are a single grapheme
it didn't seem to make a noticable difference when i was combing through a large string
jnthn The point of the ASCII/_8 variants is largely for people doing processing of really long but ASCII-based strings
e.g. bio-inf folks 14:25
timotimo maybe in the benchmark i was using all those strings were actually strands-based rather than actually containing the storage. maybe that's why the cache didn't get hit? 14:26
cygx is the grammar engine able to/supposed to be able to deal with such byte strings? 14:27
jnthn Well, the grammar engine just uses string ops
So "provided the ops support it"
timotimo the regex engine will always run "flattenropes" first, which currently forces a string into the grapheme32 storage format
jnthn We'll need to finish decouploing hash from flattening strings though
timotimo yes, very.
jnthn Once *that* is done, then a flat 8-bit string is fine for the hash
uh 14:28
for the regex engine
And it'll only be strandy things that need the flattening
cygx if byte-sized strings are supposed to become first class, they should stick around 14:31
MVMStrings.h also speculates about byte strings with synthetics - that would require the introduction of string-specific synthetic table (or rather tables for translation to the global one), correct? 14:33
jnthn No
More than in reality there'll probably not be that many synthetics 14:34
But perhaps it's better to just always go 32-bit once there are
Since if you do have synthetics, you'll probably have other non-8-bit things too
I suspect GRAPHEME_8 might want to go away
There was some thinking that you could do ASCII + use the negatives to compactly represent other codepoints 14:35
timotimo yeah, that'd totally be possible
jnthn So, for example, you can store something alphabetic like Russian in an 8-bit string
timotimo there'd have to be measurements
jnthn Which would be cool but...that's an optimization for some years into the future :) 14:36
14:39 ilmari joined
timotimo jnthn: could it be the cross thread write log has some trouble staying threadsafe? 15:04
gist.github.com/timo/a586bda606528...5370b66d8c 15:05
15:06 FROGGS joined
timotimo i'm not sure about the lifecycle of this stuff 15:07
FROGGS o/
timotimo why was spesh_graph_destroy freeing something that bytecode_finish_frame was allocating? 15:08
jnthn That's odd indeed 15:09
timotimo oh
here's a check that's probably bitrotted
jnthn I thought initial instrumentations took place under lock
timotimo "frees handlers array if different from the static frame"
if (!g->cand && g->handlers != g->sf->body.handlers) 15:10
probably wants to learn about not-yet-fully-decoded static frames?
or perhaps the first place we ask for the handlers from has to cause the sf to be fully decoded 15:11
dalek arVM: c1ada4b | timotimo++ | src/instrument/crossthreadwrite.c:
include debug_name in crossthreadwritelog

shows what type of object has been acted upon		 15:24
timotimo i'd also like to give a bit more info on the operation performed. like what attribute name was accessed for example 15:25
FROGGS @all: is here somebody who has an arm system where rakudo compiles? 16:25
(and I'm not talking about a qemu system) 16:26
(nor a chroot) 16:27
(which would be qemu)
17:31 domidumont joined
domidumont FROGGS: arm64 hang-up are getting weirder: github.com/MoarVM/MoarVM/issues/428 17:45
FROGGS domidumont: can you post the output of lsof of the time it hangs? 17:48
domidumont: another option would be to run the hanging command in gdb 18:08
so it would be: time gdb --args /usr/bin/moar --execname=./perl6-m --libpath=/usr/share/nqp/lib --libpath=/usr/lib/perl6 --libpath=. perl6.moarvm t/04-nativecall/02-simple-args.t 18:09
and when you ctrl+c, you should be in gdb IIRC and would be able to generate a backtrace
domidumont FROGGS: I've added the output of lsof in gh ticket 18:37
FROGGS: the test does not hand when run under gdb (timing issue ?) 18:38
s/hand/hang/ 18:39
FROGGS domidumont: I have no clue why it hangs :o( 18:47
domidumont can't I active a trace, logs, to have a better idea what going on when it hangs ? 18:48
FROGGS timotimo: do we have something suitable? 18:53
domidumont it blocks on epoll_pwait(5, 18:54
I can provide the whole strace output
FROGGS that'd be awesome
domidumont github won't like it :o) (~ 1800 lines) 18:56
FROGGS: done 18:57
FROGGS: I can send you a file by mail if it's more convenient for you 19:00
FROGGS no, got it in my editor
timotimo if you have a tc
you can "call MVM_dump_backtrace(tc)"
in all of your threads
FROGGS well, but it hangs
domidumont: does it help prepending "no precompilation;" to t/04-nativecall/CompileTestLib.pm ? 19:13
also, could you output "tree t/04-nativecall/.precomp/" after a hang? 19:14
(I' need to by an arm64 machine)
domidumont let me try (I'd guess yes, since the test does not hang once the precompilation has been done)
FROGGS I'd*
nwc10 FROGGS: geizhals.de/raspberry-pi-3-modell-b...00349.html -- Raspberry Pi 3 Modell B ab € 33,53 19:15
although to be fair, I think that only FreeBSD currently tries to run as 64 bit, and I don't know how stable it is 19:16
FROGGS nwc10: and 1GB ram is enough?
nwc10 Might need to swap more for 64 BIT
I think I had some swapping buidling on a Pi 2 (which is ARMv7, so only ever 32 bit) and 1G 19:17
I had an external drive for swap, to avoid wearing out the SD card
anyway, the disclaimer on this is "€ 33,53" but might be more trouble than your time is worth to get it working
FROGGS I once tried to build rakudo on an old raspberry 1 b+ with 512mb ram, and it just froze when compiling core setting 19:18
domidumont: what type of machine do you have?
nwc10 took 24 hours on a 256MB early Pi 1 B, but with an external HD 19:19
that was over a year ago
FROGGS domidumont: what we also could do is to run the test when the env var RAKUDO_MODULE_DEBUG is set to 1 19:22
domidumont same result with "noprecompilation": it does hand (but not always) 19:25
I'm using this machine : db.debian.org/machines.cgi?host=asachi 19:27
I've posted .precomp content on github 19:29
is that normal that .precomp has file even if "no precompilcation" is specified ? 19:31
FROGGS no, should be empty I think 19:32
I end up with two files less when I prepend "no precompilation;" to said file 19:34
domidumont getting a hang with RAKUDO_MODULE_DEBUG is harder ... but I got one
FROGGS awesome 19:35
would should compare a success and a fail
domidumont do you need the full trace ?
FROGGS yes
domidumont sent on github. 19:44
And that's it for today. It's getting late ...
Let me know if you need more, I'll resume tomorrow 19:45
timotimo FROGGS: when something hangs, just ctrl-c it a few times and see where it travels 20:08
FROGGS well, it is not hanging on my box, sadly
timotimo damn 20:09
FROGGS and there seems to be no output after hitting ctrl+c 20:11
timotimo in gdb, of course? 20:14
ctrl-c should drop you to the prompt where you can go through the threads and print their backtraces
FROGGS it does not hang under gdb 20:18

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!