01:48 ilbot3 joined 07:02 FROGGS joined 07:37 domidumont joined 07:42 domidumont joined
nebuchadnezzar MoarVM segfault on x32 (debian amd64 with kernel option “syscall.x32=y”) lists.alioth.debian.org/pipermail/...00983.html 08:33
Using libffi or dyncall by the way
nine nebuchadnezzar: is there any way you could give be a backtrace of that segfault? 08:39
FROGGS o/ 09:03
09:08 camelia joined 09:10 domidumont joined
nebuchadnezzar nine: sure, I'm openning a github issue to track it 09:16
nine: here github.com/MoarVM/MoarVM/issues/423 09:18
FROGGS nebuchadnezzar: would be nice to recompile moar with --debug=3 09:21
nebuchadnezzar FROGGS: ok
FROGGS this way the backtrace should show more information
nebuchadnezzar FROGGS: do you think this should be integrated in the packaging? It would produce more useful -dbgsym package? 09:26
FROGGS well, this slows things down 09:27
so it might make sense to enable this in a debug build, but not in the normal package
nebuchadnezzar ok 09:29
I have to go for now, will do that in the begining of afternoon 09:30
thanks for the advices
FROGGS nebuchadnezzar++
09:44 domidumont joined 09:57 ggoebel joined 10:07 TimToady joined 10:26 TimToady joined
domidumont nine: Hello, here are the rakudo/moar test issues I've found on armhf qemu chroot: github.com/MoarVM/MoarVM/issues/424 10:57
FROGGS hmmm, interesting 11:06
jnthn Left a note but I suspect github.com/MoarVM/MoarVM/issues/423 is a configure issue (should not build with JIT on x32) 11:10
So I suspect it explodes upon executing the first JITted function
(Due to ABI mismatch) 11:11
nwc10 [A 11:37
timotimo yes 11:38
nine Failed test 'sizeof(MyStruct)' is quite simple I dare say 11:49
The 8s in CUnion's compute_allocation_strategy are quite suspicious :) 11:50
FROGGS hmmm
nine: the eights are about bytes vs bits 11:51
nine Then it's this: /* The structure itself will be the multiple of its biggest element in size. So we keep track of that biggest element. */ 12:04
FROGGS now that's just a comment :P
nine Obviously not true if the size is 28
and the struct contains e.g. a num64 12:05
FROGGS yeah, we potentially have to guess the rules again
no, wait... are we guessing 28 or 32?
if we are guessing 28, then it might be easy to spot 12:06
I'm building debian-arm64 right now btw
nine Oh wait indeed, we're guessing 28! 12:10
expected 32 and got 28 for is nativesizeof(MyStruct), SizeofMyStruct()
FROGGS # uname -a 12:15
Linux froggs-Latitude-E6530 4.2.0-42-generic #49-Ubuntu SMP Tue Jun 28 21:26:26 UTC 2016 aarch64 GNU/Linux
\o/
nine FROGGS: I think the comment is still wrong, or rather that the comment is right and the code does not do what the comment says. Because we only track the alignment rules of the members, not their size. 12:18
And if a num64 has to be aligned at 4 byte boundaries, that's what we use to multiply.
FROGGS ohh, yeah 12:19
nine OTOH we only seem to use multiple_of for rounding up.
FROGGS if (bits / 8 > total_size) 12:20
total_size = bits / 8;
but bits is about its size, no?
wait
we are not talking about CUnion, right? 12:21
-.-
lemme try that:
if (bits / 8 > multiple_of)
multiple_of = bits / 8;
(inc CStruct.c) 12:22
nine That would bring it back in alignment (pun intended :) with the comment 12:23
FROGGS hehe
bbi30 12:28
building moar on arm64 now... 12:49
nebuchadnezzar jnthn: thanks for the advice, you were right 12:50
FROGGS nebuchadnezzar: now the question is: how we detect that to disable the jit when running Configure.pl 12:51
nebuchadnezzar the JIT support is detected by Configure.pl or should we do something in the packaging?
FROGGS we should fix Configure.pl
nebuchadnezzar ok
FROGGS: maybe detecting the size of pointer (4 bytes) and the availablity of 64 bits features (en.wikipedia.org/wiki/X32_ABI) 12:55
FROGGS yeah, I was thinking the same
nebuchadnezzar ok, for now I disable JIT explicitely in the packaging 12:59
I found paste.debian.net/858210/ from sites.google.com/site/x32abi/, I don't know if it help
13:00 cygx joined
cygx o/ 13:00
FROGGS nine: my changes pass on amd64
nine: now trying x86 and then arm64 13:01
cygx needs some help with 6model guts
background: I want to be able to call nqp::rebless on an MVMOSHandle 13:02
nebuchadnezzar FROGGS: I reopen #423 to track the Configur.pl
cygx that works, but now it fails when trying to serialize the type object for a class with that repr
cf gist.github.com/cygx/9c9fa30c43967...f3bcc31bb1 13:03
jnthn: ^
FROGGS nebuchadnezzar++
nebuchadnezzar FROGGS: on this packagers have the easy part, let's try to build and complain ;-) 13:05
FROGGS hehe 13:06
nebuchadnezzar Should we track x32 tests failures (github.com/MoarVM/MoarVM/issues/42...252422990) in #424 like with amhf or in a dedicated ticket? Some failures may be related.
I don't see what's the reponsibility of MoarVM, NQP or rakudo 13:07
FROGGS nine: it seems to be slightly more complicated than my proposed patch: github.com/MoarVM/MoarVM/commit/f2ae11b5 13:08
nebuchadnezzar: a dedicated ticket might be better
nine FROGGS: maybe you need both? 13:12
max(bits/8, align)
FROGGS no 13:16
if a struct is in an attr slot we cant to that
nebuchadnezzar Does building with --debug=3 and strip debugging symbol (they are in a dedicated package) produce a slower moar binary? 13:18
FROGGS: héhé wiki.debian.org/X32Port#Detecting_X32 13:26
jnthn cygx: Why do you need rebless on that? 14:19
cygx: I'm really, really not keen on supporting it more widely than on P6opaque
cygx jnthn: not need, but want :p
cf github.com/cygx/p6-newio/blob/master/NewIO.pm
jnthn Yes, but you also wanted to do runtime mixins to IO handles a while back, which is just no. 14:20
Long story short, any time you rebless, you deopt the whole callstack, except to actually do it right we should probably really deopt the callstacks of all running threads in the future unless we can prove we don't need to 14:22
I see a nice absence of those in github.com/cygx/p6-newio/blob/master/NewIO.pm :) 14:23
Oh, except the one rebless :P
Is there really no better design, though?
cygx a wrapper object 14:24
jnthn That'd be preferable
cygx I can do that
jnthn fwiw though, your patch to implement rebless on MVMOSHnadle does look correct. 14:25
cygx it's just a bit sad that there's all this 6model goodness, but in the end, it still comes down to wrapper objects 14:26
if I wanted that, I could do Java instead ;)
but of course the requirements for CORE stuff are a bit different...
jnthn At the end of the day, a lot of optimization - even of dynamic code - relies on it settling into relatively static behavior 14:27
I guess there may be clever things we can do with tracking "this type usually gets mixed in to" and other such things
But yes, for now it's nice if CORE stuff is written in terms of stuff we can optimize 14:28
Rather than causing opts to be backed out
About uniread and so forth, I'd been wondering a bit about passing norm => ... during open 14:29
And then things like .get and .lines might return Str or Uni 14:30
Which feels like it'd make refactoring a tad easier if you start out with Str and then realize you need more control 14:31
The default for norm being Str, but other options being Uni, NFC, NFD, etc.
cygx my reasoning was that in principle, if you take care of properly maintaining your buffers (eg the .restore-bytes method on Decoder), there's nothing stopping us from intermixing binary, grapheme-based and codepoint based IO 14:32
so the 'low-leve' methods always return a consisten type and can be used at any time
jnthn You are I assume re-using the encoder design I already posted? :)
cygx high-level things liky lines/sput/slurp are multis that accept :uni or :bin and dispatch accordingly
jnthn And partly implemented?
cygx nothing implemented so far, just playing around with the API 14:33
jnthn gist.github.com/jnthn/4b2ce730fe7b...57291de572
OK
cygx I'm starting at the top to see what the bottom needs to look like 14:34
eg I'd like for the bytes corresponding to codes/chars held back in the normalization buffer to stick around so we can losslessly call a byte-based function 14:35
jnthn urgh 14:37
That'll need quite some tracking
cygx I already did something like this at the vm level (cf github.com/MoarVM/MoarVM/pull/319 ), but I needed to unnecessarily fseek() when the buffers could just be restored 14:38
wasn't there this thing about torturing the implementor ;)
jnthn Yes, though I think this falls under "hard things possible" :P
Do you actually have some concrete use cases? 14:39
Torturing the implementor for something nobody's ever going to need is a little pointless.
cygx it's switching encodings on-the-fly on steroids 14:40
most real-world use cases I can think of can be handles by just doing binary IO and manual encoding 14:43
but from the 'artistic' perspective, I still like the idea that I do not need to care if I'm dealing with a binary, text or codepoint-based file 14:44
jnthn The real world use case that comes most immediately to mind is HTTP, but in that case you'll most likely be dealing with the data "reactively" 14:46
(e.g. it'll be arriving over the socket in chunks)
grmbl, headache...afk for a bit 14:57
14:58 domidumont joined
cygx nasty things, headaches 15:02
in my youth, I used to get hit with them fairly reguarly, so my sympathies
FROGGS nine: I guess we need more like: min(max(align, bits/8), sizeof(void *)) 15:36
nine That's 4 bytes on x32 15:41
Which I think is exactly what we currently get :) 15:42
FROGGS are you sure? bits/8 should be 8 for our use case here 16:06
so the result will be... 16:07
damn
maybe we just do min(max(align, bits/8), 8) ?
I guess that might be sane
this passes on amd64 and x86 so far: gist.github.com/FROGGS/9421a660bd7...ae34e76440 16:19
arm64 is still building rakudo
Stage parse : 666.390 16:23
heh
timotimo %) 16:24
seems like a devilishly slow device
arnsholt nine, FROGGS: The size of a struct can be pretty much whatever you damn well want it to be 16:27
For example struct { int32; char} will be 9 bytes 16:28
The *alignment* requirement is of course the alignment of its member with the larges alignof, though
OTOH struct { char; int32 } will be 8 bytes, due to the alignment requirements of the int32 16:29
16:29 synopsebot6 joined
cygx arnsholt: struct size is always a multiple of the alignment 16:30
geekosaur not entirely true, actually. struct {double, int32, double} is usually 20 not 24
cygx otherwise, you'd have to introduce padding between array elements
arnsholt geekosaur: How does that happen? Isn't double 8-byte aligned? 16:31
geekosaur ...but alignment depends on the ABI, and there are indeed ABIs where the alignment is 8 instead of 4
it is, and a special alignment is used just for that while other things are 4-byte aligned on the ABIs that work that way 16:32
arnsholt cygx: Point 16:34
FROGGS okay, then my patch is wrong 16:35
geekosaur arnsholt, you might want to look at "man offsetof" which shows what I was referring to. x86_64 may differ; their example is i386 16:38
actually my prior example was wrong because of the second double, yeh. 16:40
should have included a second int as well so the second one would show the align being 4 for those
dalek arVM/overflow_exception_mvmarray: cc71759 | timotimo++ | src/6model/reprs/MVMArray.c:
clip size at biggest possible, or throw exception for growing

when a ridiculously large size is requested, first give the exact maximum malloc and realloc will let us specify, and if we're asked to grow past that, throw an exception.
Have fun waiting for the zeroing of 2**62 or 2**63 slots ...		 16:44
geekosaur it also at least used to be possible to define an ABI where doubles were 4-byte aligned, but that came at a performance penalty (extra clocks to access unaligned values) and iirc Intel removed at least some unaligned access ability in later cpus although other unaligned accesses can still be enabled in the cpu flags/control word 16:46
geekosaur still remembering horrid SCO 3.2 kernel ABI that didn't match the default userspace ABI re alignment 16:47
dalek arVM/overflow_exception_mvmarray: 0573c98 | timotimo++ | src/6model/reprs/MVMArray.c:
use SIZE_MAX, which actually exists. also include limits.h
 16:48
timotimo ah, damn. 16:49
that'll apparently silently give you a smaller array when you ask for a specific size and not asplode at all
BBL 16:50
geekosaur welcome to the hell that is C >.>
timotimo well, no, that's not C's fault 16:51
that's how our stuff works
maybe i do throw the exception but it gets caught :P
who knows
anyway, BBL for now
this went into a branch for a good reason
FROGGS arnsholt: struct { int32; char} would be 12 bytes, no? 17:56
japhb FROGGS: How do you get 12 bytes? There's only 5 bytes worth of data there. 17:57
FROGGS hmmm, right
should be eight bytes also
no...
well yes 17:58
FROGGS reads www.catb.org/esr/structure-packing/ 18:02
cygx struct { int32; char[3]; int16; } would be 12 bytes, with one padding byte after the char[3] and two after the int16
geekosaur on x86 that is 8 bytes 18:11
sorrt, I mean the int32/char
cygx is correct about theirs, again on x86 (with its standard ABI, not weirdshit like the old SCO kernel ABI) 18:12
and this is why perl 5's stuff (and haskell's, and pretty much every other FFI out there) goes through C and asks the C compiler to report exact offsets for struct members etc. 18:14
it depends on the processor family, it depends on the ABI, there is no single hard and fast rule that is guaranteed to work everywhere. even in the modern world where you can mostly pretend only x86_64 and arm exist, there are ABI differences between POSIX systems and Windows, and every ARM generation has slight ABI differences 18:16
(and that doesn't help those of us who support big companies that still run SPARC and have no intention of getting rid of it)
(or likewise IBM POWER/PowerPC) 18:17
cygx gist.github.com/cygx/fec5e5aa03df8...5c25a74131 18:21
^ compute size/alignment if you know size/alignment of the members
(hopefully ;)) 18:22
18:28 dalek joined
FROGGS hmmm 18:41
cygx: I can try your formula
cygx: seems to work on amd64, x86 and arm64 (all linux) so far... 18:56
will check x86 and amd64 on windows now... (gcc and msvc) 18:57
though, I got another issue: 18:59
# Failed test 'returning char works'
# at t/04-nativecall/03-simple-returns.t line 19
# expected: '-103'
# got: '153'
... on arm64
is this the same that's failing on osx?
nwc10 'char' is unsigned on arm (presumably arm64 too)
cygx yup - 'char', 'signed char' and 'unsigned char' are 3 different types 19:06
two of them have the same representation, but which will depend on the ABI
FROGGS think I found the (un)signed char issue 20:09
20:10 cygx_ joined
dalek arVM: 9723396 | FROGGS++ | src/core/nativecall_dyncall.c:
make sure we mean "signed char" when we say "char"

This fixes an issue on arm64 and probably on arm and osx also.		 20:23
nebuchadnezzar In MoarVM build log I see several “warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]”, could it cause some issue on arch like x32? 20:59
dalek arVM: b2ed8e4 | FROGGS++ | src/6model/reprs/C (2 files):
fix calculating structure sizes for arm64 and others

  cygx++ for the algorithm. Tested on:
   amd64, x86, arm64 (linux, gcc)
   amd64 (windows 7 x64, msvc + gcc)
   x86 (Windows XP, msvc)		 21:06
FROGGS nebuchadnezzar: yes
nebuchadnezzar: err, cause? I guess not 21:07
21:10 Ven_ joined
nebuchadnezzar FROGGS: I'll testing your last commit on x32, is there anything I can look for without builing everything to see if --no-jit is automatically added? 21:10
ho, it looks like this has nothing to do you the jit stuff, just some failing tests ;-) 21:12
FROGGS right
the jit-detection is untouched 21:13
though, arm64 is fine now on my box
nebuchadnezzar thanks
FROGGS pleasure :o)
jnthn FROGGS++
nebuchadnezzar you fixed #424, right? 21:14
FROGGS I think so 21:15
though, gimme a se to verify
sec*
nebuchadnezzar no problem, it's time to sleep for me ;-) 21:16
FROGGS well then, off to bed :o) 21:17
gnight nebuchadnezzar
lizmat FROGGS: fwiw, looks like compiling MoarVM is a lot quieter now on OS X 21:40
FROGGS really? hmmm, not sure it is due to my patches... 21:41
lizmat Files=45, Tests=609, 8 wallclock secs ( 0.26 usr 0.11 sys + 32.71 cusr 3.68 csys = 36.76 CPU) # make test
FROGGS but it is good anyway :o)
timotimo github.com/libtom/libtommath/pull/56 - something about mp_rand
lizmat FROGGS: ah, it was already quieter before 21:42
timotimo: yup, looks like the problem to me
timotimo not entirely sure what exact defines we have set in our libtommath configuration when we build moar 21:44
lizmat neither, but it looks like skids was already on it 21:45
Files=1146, Tests=53273, 250 wallclock secs (14.49 usr 4.64 sys + 1504.16 cusr 135.98 csys = 1659.27 CPU) # spectest 21:47
timotimo i just posted to the mailing list about this
lizmat on a non-cold machine, tomorrow will have better numbers
timotimo oh! 21:50
it was skids who opened that pull request!
now i notice what you meant when you said skids is on it
geekosaur recalls perl<6 once upon a time had special tests of rand() because some systems lied about the range and others were just much smaller than expected, etc. 21:55
timotimo ah 21:57
geekosaur and I see that pull mentions some linux only using half the range (no negative numbers) 21:58
siiigh.
dalek arVM: c8d9ea6 | FROGGS++ | Configure.pl:
disable JIT on x32, resolves #423
 21:59
22:04 FROGGS joined
dalek arVM: 082c989 | FROGGS++ | Configure.pl:
fix pointer size unit in configure message
 22:40
FROGGS building a debian-x32 now while going to bed... 22:42
gnight
timotimo good night! 22:45

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!