00:04 SmokeMachine_ joined 00:06 jnthn_ joined 00:07 hoelzro joined 00:11 eaterof joined 00:13 FROGGS joined, robertle joined, camelia joined, ChanServ joined, nine joined 00:19 yoleaux joined
MasterDuke timotimo: now a segv with your branch and MVM_SPESH_INLINE_DISABLE=1 and --profile-stage=parse 00:34
small backtrace though
#0 MVM_interp_run (tc=tc@entry=0x555555758c60, initial_invoke=0x1d34, invoke_data=0x0) at src/core/interp.c:5929 #1 0x00007ffff7734555 in MVM_vm_run_file (instance=0x555555758260, filename=<optimized out>) at src/moar.c:407 #2 0x0000555555555312 in main (argc=15, argv=0x7fffffffe338) at src/main.c:299
timotimo does that happen to be the extop thing?
MasterDuke yeah 00:35
in OP_CALL_EXTOP:
00:36 ilmari[m] joined
MasterDuke timotimo: any other debugging i can do? 00:44
timotimo that's a bug liz showed me earlier 00:59
i don't yet know how it happens
MasterDuke ok 01:01
01:08 AlexDaniel` joined, wictory[m] joined 01:12 nativecallable6 joined, coverable6 joined, unicodable6 joined, committable6 joined, undersightable6 joined, benchable6 joined, squashable6 joined, shareable6 joined 01:16 quotable6 joined, reportable6 joined, greppable6 joined 01:28 ggoebel joined, nebuchadnezzar joined, mtj_ joined, nwc10 joined 01:30 FROGGS_ joined 01:31 greppable6 joined, reportable6 joined 01:34 eater joined 01:35 reportable6 joined, greppable6 joined 01:56 ilbot3 joined 01:58 ilmari[m] joined 02:28 evalable6 joined, undersightable6 joined, shareable6 joined, squashable6 joined 02:31 wictory[m] joined, AlexDaniel` joined 03:49 yoleaux joined 05:14 coverable6 joined 05:56 domidumont joined 06:27 lizmat_ joined 06:58 Ven`` joined
nine Kaiepi: your tests are all about wchar_t* and CArray[wchar_t]. With "full support" I mean also being able to pass around a wchar_t as in take_a_character(wchar_t single_char) { ... } or returning one wchar_t 07:08
Kaiepi oh
passing single wchar_t works 07:09
the issue is CArray[wchar_t] doesn't seem to work
nine I think your nqp/rakudo changes would help a bit in understanding why (wchar_t does work) 07:12
Kaiepi alright, i'll put up the pullreqs for them 07:13
nine My guess is that you map wchar_t to one of the existing integer types of the appropriate size. That is why we don't need more changes. 07:28
same as size_t and ssize_t 07:29
Kaiepi yeah, i do
it'll be a second before i can get the nqp pullreq up since node's really finicky on my system for testing the js vm 07:30
done 07:46
07:47 Ven`` joined
nine Kaiepi: in github.com/MoarVM/MoarVM/pull/824#...-377455405 you didn't allocate any memory for bar 07:54
Kaiepi oh 07:57
lol
samcv i'm thinking ((INTEGER & 0xffffff80) + 0x80) & (0xffffff80-1) should work the same on big endian as little endian? 07:59
nine samcv: me, too 08:00
samcv cool. i'm super excited now. making collapsing strands 1.5-4x faster should speed up a ton of use cases 08:03
it's probably more like 2-4x actually
nine samcv: sounds amazing :) 08:05
Kaiepi alright, i got carray to work
Geth MoarVM: samcv++ created pull request #830:
Speed up collapse_strands 2-4x by full rewrite targeting SIMD optims		 09:10
timotimo now that we use SIMD, we'll potentially have to be careful about our binary packages and compatibility with systems that don't have the required cpu features 09:17
samcv timotimo: well -O3 enables it by default. at least certain ones.
timotimo right, but if we distribute a package that the compiler put AVX into, it'll crash violently on older machines :)
samcv some of our loops already did vectorize but not sure how notable any of them are
so not a new problem, but def something to take into consideration 09:18
timotimo there's surely libraries out there that can help with this problem, for example by runtime cpuid inspection? 09:19
samcv yeah
i mean ff and chrome both require sse2
timotimo yeah, ok, sse2 is rather old at this point :D
samcv so at minimum that seems sane. 09:20
hah
i'd have to check those loops if i disable some newer ones. i was suprised it did it by itself
but i guess it's an -O3 thing 09:21
timotimo could be
samcv or something... compilers are complex
i wonder what their baseline profile is
timotimo i used to have -march=native on my configure line for the longest time
so it's actually faster to go through the whole string first with can_be_8bit and then doing the copy? rather than doing it in big chunks? 09:22
samcv no clue about chunks. but it's faster even without chunking
timotimo hm? 09:23
samcv maybe for very very long strings chunking could make sense. that could be done later. but even on ginormous strings the change is much faster
as in. it checks the whole strand at a time
timotimo i thought it could be important to go over the same piece of data twice while it's still in the cache
samcv hm. may or may not be faster 09:24
timotimo my cache seems to be 6 mibs big, i take it your ginormous strings were noticably longer than that?
samcv yeah
i guess it's something we could test later. we'd have to start and stop copying many times though. no clue how that would affect it 09:25
i mean you're asking if it'd be faster to check X graphemes to see if they fit in 8 bit. then copy those 8bits
continue until we're done or we find a section that won't fit in 8bits 09:26
though i'm guessing with most normal size strings there won't be any difference between the two options if it stays in cache
jnthn_ morning o/ 09:31
samcv good *
timotimo can you graph the time it takes to collapse strands of different total sizes and see if there's any point where the characteristics change noticably? that'd be a good indicator for cache-related performance i'd think 09:35
samcv timotimo: if you're curious, clang compiles SSE2, SSE1, AVX BMI2 AVX2 BMI. no clue what it does
detected with this github.com/pkgw/elfx86exts 09:36
Geth MoarVM/master: 14 commits pushed by (Jeremy Studer)++, (Jonathan Worthington)++
review: github.com/MoarVM/MoarVM/compare/a...8d2fc9daf9		 09:37
timotimo oh, i think i might have added something to 3rdparty without adding it to LICENSE 09:38
09:39 Ven` joined
samcv jnthn: i can probalby make a static inline function to get the length left in the current strand, but i use the struct's values very extensively since i need the position and the end by themselves as well and i read from the current strand's blob as well 09:51
what is neat is after having worked on this i was able to alter some other loops we have and vectorize those. like the .flip function can vectorize that copying and probably others. not sure where else has code that can be worked to get it vectorized (and has a performance impact enough for it to be worth doing) 09:57
jnthn samcv: Hmmm... I'm a bit sad that having carefully kept the grapheme iterator struct hidden behind a set of (inline) functions to see it poked into, is all 10:00
It'll only make it harder to do further refactors/improvemnets in the future. 10:01
timotimo hm, it's not enough to copy these values out and write them back when we hand the struct off to another sub?
samcv i understand your reasoning and agree that is best, but in this case it seems hard to avoid since i'm dealing with offsets of the blobs themselves so i can memcpy from them
10:17 Ven`` joined
dogbert17 o/ 10:19
jnthn o/ 10:22
dogbert17 jnthn: the fixes you made yesterday, do you have them in a branch somewhere? 10:23
jnthn yes, only a local one 10:24
I've got a bunch of other debugging changes too
And didn't yet spectest those other fixes to make sure they're fine 10:25
dogbert17 ok
jnthn samcv: Poking into MVMString isn't my concern - we've always had that as a data structure that stuff under src/strings/ can deal with directly. It's only the gi that I'm thinking about here. 10:26
dogbert17 jnthn: first gist of the day, is the one or a false positive :) gist.github.com/dogbert17/21d8f4fc...4b9d9cbe2d 10:28
*is it the one 10:29
jnthn Well, that's the eventual place we explode
But it turns out instrumenting add_to_worklist to look for this case also spots such a thing 10:30
dogbert17 aha, so you're getting closer
jnthn That it's a use after free or a buffer overrun are both bogus, however 10:31
Sometimes the junk values just are within such a range that it detects them as such
Other times it says it's a wild access, which is more correct# 10:32
I've eliminated it being STable chaining, which is one of the members of the union where the SC lives
That doesn't ever get triggered in this program
That means we have something in a gen2 region that somehow has got a forwarder pointer
That should never happen 10:33
dogbert17 you're working up to a blog post here :) 10:34
jnthn Also, the problem my addition to worklist_add detects is indeed the same bad pointer that shows up later 10:37
dogbert17 so where does that bad pointer come from 10:42
10:43 Ven`` joined
jnthn Hm, another alternative is that it's not actually a forwarder, but rather chained in the gen2 freelist 10:52
dogbert17 sigh, valgrind really isn't helping here 10:56
jnthn No, I don't think it easily can 10:58
Nor ASAN
Wrong kind of memory problem
dogbert17 I got another error yesterday, in 'pass_work_item', but then I had changed the program under investigation slighly
jnthn m: say 4244 - 4096
camelia 148
jnthn OK, we're marking something that's on the GC free list. Wow. 10:59
11:00 Ven`` joined
jnthn Finally, something that looks closer to a clue... 11:08
#0 MVM_panic (exitCode=1, messageFormat=0x7ffff612b160 "Collectable %p in a gen2 freelist accessed") at src/core/exceptions.c:827
#1 0x00007ffff5f05166 in deserialize_closure (tc=0x6170014b5380, reader=0x6120000c34c0, i=21) at src/6model/serialization.c:2330
Yup, think I got it 11:14
And it's a silly bug 11:15
dogbert17 nice digging. Also agree that many hard to find bugs turn out to be simple mistakes in the end. 11:19
jnthn OK, spectest while I lunch :) bbiab 11:21
dogbert17 jnthn++, AlexDaniel will be ecstatic :) 11:24
jnthn grr, spectest fails, but now I realize I'm some way behind master in both NQP and Rakudo 12:14
Wow, I accidentally had comitted the small nursery and GC debug mode on 12:21
And it shows up a test failure with native calling
dogbert17 an unexpected bonus problem :) 12:22
jnthn And, thankfully, much easier to fix :) 12:27
dogbert17 there might be more of those unless your recent fixes has removed them all 12:31
timotimo whoops, that'll increase performance though when the nursery gets big again :D
lizmat which commit was that ? 12:32
jnthn A local one that already got rebased with the accidental changes removed :) 12:33
timotimo ooooh
lizmat ah, ok
jnthn Just meant that my lunchtime spectest was run under stress setttings rather than normal
lizmat
.oO( getting our hopes up, eh?)
Geth MoarVM: 47a520300f | (Jonathan Worthington)++ | 5 files
Rename mutex for more clarity over what it covers
 12:36
MoarVM: a7bb6f0d27 | (Jonathan Worthington)++ | 2 files
Use FSA for all_scs list

While the updates to it are done under lock, it may be read without one. That can be made safe by only freeing the memory at a safepoint rather than immediately.
MoarVM: be3ef87218 | (Jonathan Worthington)++ | src/core/frame.c
Add missing MVMROOTing of a frame
MoarVM: 6ba2d8d149 | (Jonathan Worthington)++ | src/core/frame.c
Simplify rooting by using MVMROOT2
MoarVM: df4c94ca38 | (Jonathan Worthington)++ | 4 files
Spot use of an item already put in a gen2 freelist

A bug currently under investigation results in this happening. Add it to our set of debug checks, so we can find such problems faster in the future.
MoarVM: ce38e35f27 | (Jonathan Worthington)++ | src/6model/reprs/SCRef.c
Fix missing GC mark of deserializer contexts list

This was the root cause of github.com/rakudo/rakudo/issues/1660
MoarVM: 85fc758ce9 | (Jonathan Worthington)++ | src/6model/reprs/CStruct.c
Missing write barrier enforcement in CStruct
12:45 releasable6 joined 12:46 Ven`` joined
dogbert17 wonders have many reported bugs these fixes resolved 12:49
12:53 travis-ci joined
travis-ci MoarVM build passed. Jonathan Worthington 'Missing write barrier enforcement in CStruct' 12:53
travis-ci.org/MoarVM/MoarVM/builds/360254039 github.com/MoarVM/MoarVM/compare/d...fc758ce976
12:53 travis-ci left
lizmat time to bump NQP and Rakudo ? 12:54
jnthn Don't mind...I'm running spectest again under the GC stress now, to see if there was anything of interest there or if it was simply me being a bit behind on HEAD 12:57
hah, see it was done anyway :)
timotimo wouldn't it be nice if that fixed some of the problems i was seeing with the profiler .. :) 12:58
jnthn Yes. :P 12:59
timotimo oh, huh, the profiler example also explodes violently on master, i thought it was new with my branch 13:24
well, that makes it more interesting
lizmat hmmm... I got a hang on t/spec/S17-supply/syntax.t on the last test just now, haven't seen that in a long time 13:27
can't reproduce it :- 13:28
(
jnthn I can
With GC stressing
Also taking that test out of the spectest file and running on its own also hangs quickly in such a way
(That's what I've been working on for the last 30 mins or so)
lizmat ok, so *I* don't need to worry about it right now
jnthn It's something about Supplier::Preserving, it seems 13:29
urgh, yes
lizmat well, I'm working on ThreadPoolScheduler.cue for the common case ( .use { }, :catch )
jnthn nice :) 13:30
lizmat makes it about 5x as fast for the common case
jnthn Goodness, that's a bit 13:31
lizmat and about 25% fewer allocations
yeah, lot's of named parameters that we don't use in most cases
but it seems I borked the non-common cases 13:32
will look at that after I get back from some cycling&
jnthn Enjoy
Yeah, the problem is seemingly that Supplier::Preserving's reply mode will emit under a blocking rather than async lock 13:33
And that test depends on cooperation 13:34
But it's not clear at all that it's so simple as "just change the type of the lock"
No, that actually makes it worse :) 13:35
dogbert17 is trying to figure out if AlexDaniel's whateverable SEGV is gone now 13:45
jnthn: this stacktrace look quite familiar, doesn't it: gist.github.com/AlexDaniel/5e0f9e0...adc4c2c4ff 13:51
it's from github.com/rakudo/rakudo/issues/1278 13:52
jnthn Very
Yeah, looks identical problem
*like an
dogbert17 :) you might have made AlexDaniel very happy today
jnthn And so presumably covered by the same fix
Damn, this Supplier::Preserving change is a small headache 13:53
dogbert17 I ran his testsuite several times, no SEGV
jnthn Cool!
Decided to give up patching it and write out the semantics I think it needs and then to implement that, and see if it helps :)
13:54 robertle_ joined 14:00 Kaiepi joined 14:02 robertle_ joined
dogbert17 it's possible to get a MoarVM Panic if t/04-nativecall/11-cpp.t is stressed hard enough: gist.github.com/dogbert17/34ca1772...2757f4f23a 14:06
jnthn It may want a similar patch to my CStruct one, if it's a CPPStruct bug
dogbert17 there are a few 'is repr<CPPStruct>' in the test file 14:09
14:17 Ven`` joined 14:18 ggoebel joined 14:23 Kaiepi joined
dogbert17 hehm I tried to implement your CStruct fix into CPPStruct and it actually seems to work 14:26
14:35 zakharyas joined 14:52 Kaiepi joined
dogbert17 got another error when stressing t/spec/S32-exceptions/misc.t: MoarVM panic: Must not GC when in the specializer/JIT 14:57
jnthn: should I PR the CPPStruct.c changes, I just did the same as you did in CStruct.c and it seemed to resolve the problem 15:16
15:17 Kaiepi joined
Kaiepi is there a way to force moar to coredump? 15:17
jnthn dogbert17: Yes 15:19
Geth MoarVM: dogbert17++ created pull request #831:
Add missing write barrier enforcement in CPPStruct		 15:39
dogbert17 here's, a slightly crappy, gist from gdb showing the 'Must not GC when in the specializer/JIT' error :gist.github.com/dogbert17/41425fc5...eac9d42d1a 15:46
dogbert17 goes for a walk & 15:47
15:49 domidumont joined
timotimo *cough* *cough* 15:56
that's still that one bug
i still didn't fix it >_>
ok, i'll engineer a feature to shove off an MVMSpeshPlanned to the future when hopefully the wval in question has been resolved 15:57
or alternatively
push that wval's data into a little queue and in between speshes, where the spesh thread stops to participate in gc anyway, just resolve that object before attempting another time
yeah, that should be sane and have only a miner impact on performance 15:58
16:34 AlexDaniel joined 17:00 Geth joined
Geth MoarVM/dont_gc_in_spesh: ecea2098ba | (Timo Paulssen)++ | 8 files
spesh worker: when wval fix fails, retry a few times
 17:31
timotimo dogbert17: please see if you can get the S32-exceptions/misc.t to still explode with this branch
AlexDaniel squashable6: next 17:35
squashable6 AlexDaniel, āš šŸ• Next SQUASHathon in 6 days and ā‰ˆ16 hours (2018-04-07 UTC-12āŒUTC+14). See github.com/rakudo/rakudo/wiki/Mont...Squash-Day
timotimo dogbert17: how small is your nursery?
m: say 4194304 +> 8
camelia 16384
timotimo this small enough?
ah, yes, i do get the crash on master 17:37
i don't get any output of my thing triggering 17:42
dogbert17 timotimo: I believe that I had 64k 17:47
17:49 nativecallable6 joined, notable6 joined, greppable6 joined, releasable6 joined, bloatable6 joined, committable6 joined, evalable6 joined, coverable6 joined, quotable6 joined, reportable6 joined, unicodable6 joined, bisectable6 joined, benchable6 joined, squashable6 joined, shareable6 joined 17:50 statisfiable6 joined
timotimo what's wrong with my signalling ... 17:51
dogbert17 signalable6: what is timotimo doing wrong 17:52
:)
timotimo any other files that trigger that crash? 17:54
dogbert17 there used to be, let me think ... 17:56
hmm, can't seem to find it, grr 18:01
timotimo oh, the code is still wrong, as it doesn't properly mark the scs before trying to resolve the objects, heh. 18:04
gah, i hate how MVMROOT messes up gdb 18:09
ok, it does happen, it runs, it seems to be good 18:15
Geth MoarVM/dont_gc_in_spesh: b5609f8372 | (Timo Paulssen)++ | 8 files
spesh worker: when wval fix fails, retry a few times
 18:16
timotimo dogbert17: this seems to work now 18:17
dogbert17 yay, timotimo++ 18:22
18:24 zakharyas joined 18:41 mojca1 joined
mojca1 Does anyone have any idea what could go wrong in trac.macports.org/ticket/56190 ? 18:43
It's strange that MoarVM built correctly on our CI, including Mac OS X 10.5/ppc
timotimo oh damn!
huh? but i did mark it MVM_PUBLIC 18:44
mojca1 It compiled for all our systems on the CI (10.5, 10.6 ā€¦ 10.13), I don't actually know how to reproduce it (not that I have 10.12 at disposal) 18:53
timotimo if someone can reproduce it, it'd be nice to have the makefile and all the flags that were passed to the compiler 18:54
mojca1 I asked the reporter to file a ticket and then provide more info on request, I cannot really help if it's not obvious how 18:56
he was using an external drive apparently, but I don't know if that makes any difference
thank you 18:57
Kaiepi why are the mux inside the loop instead of outside here? github.com/MoarVM/MoarVM/blob/mast...ile.c#L143 19:31
19:32 mojca joined
jnthn Kaiepi: Probably they don't need to be, though I guess EINTR is rare enough that it matters little 19:45
github.com/MoarVM/MoarVM/blob/mast...file.c#L73 has them outside such a loop, for example 19:46
19:56 Kaypie joined
dogbert17 jnthn: do you think that one of your fixes today nailed github.com/MoarVM/MoarVM/issues/777 ? 19:58
jnthn dogbert17: yes 19:59
20:00 Kaiepi joined
dogbert17 guess we can close it then given that it was the tests which uncovered the problem :) 20:01
jnthn yay :) 20:02
20:03 Kaiepi joined
dogbert17 done :) 20:03
Geth MoarVM: e81a0ee273 | (Jan-Olof Hendig)++ | src/6model/reprs/CPPStruct.c
Add missing write barrier enforcement in CPPStruct

This was done earlier for CStruct, by jnthn++, in commit 85fc758ce976. Stress testing of t/04-nativecall/11-cpp.t showed that the same fix had to be applied to CPPStruct as well.		 20:05
MoarVM: 0aaea3c4ff | (Jonathan Worthington)++ (committed using GitHub Web editor) | src/6model/reprs/CPPStruct.c
Merge pull request #831 from dogbert17/fix-wb-enforcement-in-CPPStruct

Add missing write barrier enforcement in CPPStruct
dogbert17 jnthn thx 20:08
I found one item which still seems a bit buggy: gist.github.com/dogbert17/e1007f3b...31d9f85422
when retesting github.com/MoarVM/MoarVM/issues/728 20:09
when ASAN gets grumpy it's always on test #26 20:11
20:21 mojca1 joined 20:23 Kaiepi joined 20:26 Kaiepi joined 20:31 mojca joined
samcv timotimo: regarding whether loops vectorize or not. we can use #pragma clang loop vectorize(enable) interleave(enable) and set a clang option to warn (or it may warn actualy by default if it fails in vectorizing) 20:44
20:53 Kaiepi joined
MasterDuke samcv: what does gcc do with the loops? 21:52
samcv it probably vectorizes them 22:07
also i just tested. it still vectorizes even if i only let it uses sse1 and sse2 22:10
MasterDuke in clang or gcc?
samcv clang
i am checking now though 22:13
ack sadly it seems to not be vectorizing them. though the changes are still slightly faster than what we were doing before if there's no vectorization 22:16
MasterDuke any way to hint them
samcv i'm trying to figure that out now 22:17
src/strings/ops.c:276:23: note: not consecutive access *_18 = _26; 22:23
what... it's totally consecutive
result8[result_pos++] = active_blob[i++];
maybe it's not unrolling any loops? 22:25

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!