00:16 stmuk_ joined 02:48 ilbot3 joined 06:44 domidumont joined 06:48 domidumont joined 06:50 domidumont joined 06:52 domidumont joined 08:23 zakharyas joined 08:59 brrt joined
brrt \o #moarvm 08:59
nwc10 o/
brrt how are things? 09:00
nwc10 OK.
brrt hmm, okay :-) 09:10
me, I'm happy, because I (conceptually, at least) solved a problem of unreferenceable references 09:11
masak .oO( next, I will move on to squaring the circle! ) 09:13
brrt well, it is a bit more specific than that :-) 09:14
roughly; synthetic tiles don't live in the big tile array before their final step, so they can't be referenced by index into that big tile array; however, we can reference them out-of-band, and that will DWIM 09:15
as in, I think it is safe to do so
masak I'm intrigued by the use of the word "synthetic" 09:24
that's something I use for 007 ASTs, too
(things that are parsed from text are "natural"; things that are created in vitro as AST objects with no connection to source text are "synthetic") 09:25
brrt well, it makes sense 09:38
synthetic is basically used as standing for 'artificially created', which is of course not entirely what it means 09:39
that usage probably comes from synthetic organic chemistry, in which things are synthesized as in 'put together'
(and the reason to reference them in the first place is to ensure they are assigned the correct registers) 09:43
masak on average, a macro will want to freely mix parsed and synthetic ASTs. I find macros come to their full strength when one does. 09:45
the tricky detail is that (in Perl 6) a lot of checking happens during parse, so parsed ASTs are automatically safer/saner than synthetic ones. 09:46
two examples: you could create a synthetic Q::Identifier whose name is the empty string. can't happen with a parsed one.
you could create a synthetic AST containing a reference cycle.
jnthn morning, #moarvm 09:47
brrt morning jnthn
hmm, yes, you could
masak a lot of things like that will have to be checked during macro expansion, when the synthetic AST is inserted into the program.
morning, jnthn -- Perl 6 day today? :)
brrt tbh, perl6 *ought* not to do the checking at parse time, necessarily
jnthn masak: Yup :) 09:48
masak brrt: there are very interesting tensions there, for sure
brrt or rather, the checking element could be implemented as a tree iterator and then work both during parsing and during macro expansion 09:49
masak heh
we'll get there in the end :>
it's exactly questions like these that 007 has (quietly) helped delve into in the past two years
jnthn I guess it boils down to syntactic vs. semantic checks 09:50
Where syntactic ones living in the grammar are no bother because you can't construct a "syntactically invalid" AST node 09:51
masak it's more like -- the textual medium puts up certain natural text-based limits
there's nothing *a priori* wrong with an identifier called "" or an identifier called "42 LULZ ☠"
it's just that the rules of parsing disallow them
jnthn Right, but even there we've got escape hatches for most things 09:52
masak (it's been tempting sometimes to implement gensyms using disallowed-by-parsing names)
jnthn m: class ::("42 LULZ ☠") { }.^name.say
camelia rakudo-moar ee8ae9: OUTPUT«42 LULZ ☠␤»
masak yes, good point
this all comes back to Perl not liking arbitrary limits :P 09:53
masak .oO( what do you *mean* I can't call my class that!? )
masak .oO( I'll *show* you weird! )
brrt has had the 'but that is evil' discussion quite a few times in the last few months 09:54
'why do you hate code sanity' <- a literal question I've been asked 09:55
jnthn o.O :)
masak brrt: Perl mentality is a minority viewpoint there, which sometimes makes those discussions hard. 09:56
brrt well, in fairness, maybe sometimes I have suggested insane ideas, even for perl
but yeah
timotimo o/ 09:57
masak brrt: the rest of the world tends to think "if something is dangerous even though it's powerful and expressive, we should avoid it". Perl people tend to do the opposite: "if something is powerful and expressive even though it is dangerous, we should explore it"
brrt true 10:02
jnthn is continuing on with his string flattening / hash cleanup, fwiw
Which should make us less combustible
brrt the counterargument is usually 'but what about my colleagues'
which is less about 'my colleagues are idiots' and more about 'the bandwidth from me to my colleagues is really limited' 10:03
timotimo when we have "stores int8 chars", we can add a "doesn't own its memory" flag and point directly into the string heap :3
masak brrt: similar to the discussion about private members in classes
brrt on the other other hand, I think that expecting code to be the sole communicator of your ideas about the software is a rather limited viewpoint 10:04
jnthn fwiw, I've seen the "what about my colleagues" thing pop up discussing such insane horrors as type inference and lambda expressions in C# :)
masak brrt: it's not because colleagues are inherently power abusers; it's because it's a sane way to abstract things and expose the important bits
brrt that sanity is at least partially a convention, though
masak brrt: not the sole communicator -- but I'd say more than people in general expect
jnthn Sometimes, it's just another case of "I don't like change/new things" 10:05
brrt hmmm... I find the opposite, really
masak brrt: "the hidden assumption when asking people to write intentful code is that people have intents" :/
brrt i mean, most programmers I've met seem to take a code-first viewpoint to fairly extreme lengths
hehehe, that is true
jnthn m: for ^10000 { my $a = ("A".."Z").pick x 1000; for ^4 { start { my %h; %h{$a} = 42; } } } 10:07
camelia rakudo-moar ee8ae9: OUTPUT«MoarVM panic: Memory allocation failed; could not allocate 4 bytes␤»
jnthn aww 10:08
Anyway, that reliably triggers the thing I'm fixing, so will become my test csae :)
(It doesn't with camelia because we ulimit things so far we can't start up a full pool of threads)
masak dare I run that locally? :) 10:09
whoa, segfault!
jnthn Right :)
"Glad" to see my test case works on somebody elses machine too 10:10
masak the happiness is mutual
timotimo that's not the kind of crash i'd expect from string flattening
jnthn My $dayjob prototype that works rather nicely occasionally combusts because of this
timotimo: gdb it ;)
nwc10 ASAN does not reward you with full-on technicolor barfage :-/ 10:11
merely
==12134==ERROR: AddressSanitizer: attempting double-free on 0x603000730030 in thread T16:
jnthn Yup
The rehash branch is the refactoring to let me fix this in one place
So the task now is to fix it
masak a "double-free" is like Swedes being able to take vacation and sick leave simultaneously, aye? :P
timotimo fair enough, just totally b0rks the heap
jnthn: also make sure you transplant the "if the key is a null string, don't try to asplode" fix from master a few days ago 10:13
'cuz we're able to get an actually null pointer in there if we have an uninitialized "str" attribute in a class
jnthn Ah, maybe I should just rebase this branch on master before continuing
timotimo not sure that'll help 10:14
nine jnthn: camelia is ulimited to 80 processes and 1 GiB of virtual memory. Shouldn't that be enough?
jnthn nine: I'd have thought so
The default max thread pool size is 16 and they can each carve out up to 8MB of nursery 10:15
timotimo jnthn: actually, i can just put that fix back in when you merge into master
jnthn Which is only 128MB on top of the Rakudo base memory
timotimo not a big deal at all
jnthn So 1GB should really be enough
10:15 camelia joined
jnthn timotimo: bah, then i'll just have a merge conflict later rathe than now :) 10:15
nine m: for ^10000 { my $a = ("A".."Z").pick x 1000; for ^4 { start { my %h; %h{$a} = 42; } } }
camelia rakudo-moar ee8ae9: OUTPUT«(signal ABRT)*** Error in `/home/camelia/rakudo-m-inst-2/bin/moar': double free or corruption (fasttop): 0x0000000004a283a0 ***␤======= Backtrace: =========␤/lib64/libc.so.6(+0x727df)[0x7f9a9f6217df]␤/lib64/libc.so.6(+0x7804e)[0x7f9a9f62704e]␤/lib6…» 10:16
nwc10 and a merge conflict that's obvious in history, instead of one that's tidied away
jnthn nine++
nine Seems to be the ulimit indeed. Strange.
timotimo OK
nine Though one has to remember that virtual memory does not mean used memory. Could be mmaped files or unused but allocated memory 10:17
timotimo quite.
jnthn timotimo: I trust you added a test to cover the thing you fixed, so spectest should be enough to be sure my rebase is OK? 10:22
10:22 camelia joined
nine m: for ^10000 { my $a = ("A".."Z").pick x 1000; for ^4 { start { my %h; %h{$a} = 42; } } } 10:22
timotimo i didn't because i didn't want to force a version bump, but i can do it now because versions have been bumped in the mean time
because it gave us a segfault and i didn't want test files to segfault
camelia rakudo-moar ee8ae9: OUTPUT«(signal SEGV)» 10:23
timotimo now to find what file it ought to go into ...
nine That's now with 2 GiB ov virtual memory
dalek arVM/rehash: b687df8 | jnthn++ | src/6model/reprs/MVMHash.h:
Remove a couple of now-unused macros.
 10:25
arVM/rehash: edd3bbf | jnthn++ | src/6model/reprs/MVMHash.h:
Remove unused macro parameter.

Push MVMString awareness deeper into hash guts.
In preparation for eliminating the need for MVM_string_flatten.
jnthn There's the rebase
10:26 dalek joined
timotimo the tests in S32-hash are not really numerous ... 10:27
and none of the existing files there seems to cover just doing an assignment that can asplode
could make a new file in integration/ :\ 10:28
maybe it fits into weird-errors.t?
jnthn weird-errors.t is getting a bit...unweildy :) 10:36
timotimo compiles some rakudo to make sure the test actually fails before the moar patch 10:37
# Failed test 'using a null string to access a hash does not segfault' 10:40
ok 31 - using a null string to access a hash does not segfault 10:42
yay
the test file is pushed
jnthn Thanks! 10:47
timotimo hmm. if we go ahead with making strings storable in int8 codepoint buffers and such, we'll have to make sure these representations are actually unambiguous, right? 11:19
or does uthash allow us to build a "compare keys" function that can handle multiple different representations? 11:20
also, when we do implement the "GC synthetic codepoints when the table becomes too big" thing, we might end up having to re-hash all our keys ... well, only if we compact synthetic codepoints rather than just making a free-list 11:26
jnthn Doubt we'll compact 11:28
timotimo that makes it simpler
jnthn What we'll have by the end of this refactoring will not really be uthash any more, I suspect 11:29
timotimo hehe.
maybe when we have a hoard-like allocator we'd be interested in allocating our hash tables with that 11:34
nwc10 jnthn: origin/rehash doesn't change any spectests for me 11:37
reward yourself with lunch! :-)
timotimo looking at my current trajectory and energy levels, i won't get a hoard-like thing into moar before the next release, but this is the last release before we have a big influx of eyes from the advent calendar :| 11:39
dalek arVM/rehash: 2410ff3 | jnthn++ | / (3 files):
Implement hashing that will not need flattening.

We still need it for comparison, which will need fixing up next.		 12:02
timotimo cool 12:04
12:04 ggoebel joined
nwc10 jnthn: origin/rehash doesn't change any spectests for me 12:35
brrt what is a hoard allocator? 12:36
timotimo hoard is the name of an allocator that is multithreaded and handles producer-consumer patterns better than others
it's based on giving back mostly-empty pages to the shared heap for other threads to take 12:37
jnthn lunch & 12:38
nwc10 jnthn++
jnthn nwc10: Next patch will be The Big One :)
timotimo The Big Patch Theory
nwc10 best eat lunch first then
brrt jnthn++ cool 12:40
12:48 domidumont joined
lizmat is the plan to merge this before or after the release ? 13:01
jnthn Before provided it spectests clean, given how nasty the thing it's fixing is... 13:14
yoleaux2 12:49Z <MasterDuke_> jnthn: do you have any thoughts on my line directive PR github.com/rakudo/rakudo/pull/919 and Zoffix's suggestion?
jnthn Thing with hashes is they're used so widely, if you break something it's likely going to show up rather quickly 13:16
NQP built/tested fine. Rakudo build in progress :) 13:26
lizmat jnthn++ 13:32
would this make things faster ?
jnthn Hard to say 13:33
Though I think I'll be able to get an 8-byte memory reduction per hash key
And not having to turn all strings that get hashed into 32-bit wide ones would also save memory 13:34
And less memory use = better cache use = faster
The hash computation itself may be marginally slower
dalek arVM/rehash: 70fa980 | jnthn++ | / (10 files):
Use string comparison to compare hash keys.

This requires us to store an MVMString * in the key slot of the hash rather than a pointer to memory. That in turn means we need to make sure this pointer is known to the GC, which is what makes up most of this commit.
arVM/rehash: ad25deb | jnthn++ | 3rdparty/uthash.h:
Avoid dupe setting of string hash cache.

It's set at the point of computation now.		 13:46
arVM/rehash: 9d0a2f2 | jnthn++ | src/6model/reprs/MVMHash.h:
Eliminate use of MVM_string_flatten in hashes.

Which means strings become truly immutable, resolving a longstanding thread safety bug.
jnthn That thing that crashed sub-second reliably has now been running for 5 minutes solid here, no problems :) 13:48
timotimo \o/ 13:52
14:14 diakopter joined
dalek arVM/rehash: 052ea53 | jnthn++ | src/ (5 files):
Eliminate duplicate storage of hash key.

This shaves a pointer off each entry in an NQP or Perl 6 hash.		 14:16
arVM/rehash: bc6e933 | jnthn++ | src/6model/reprs/MVMHash. (2 files):
Missing nullness checking.
 14:21
14:22 domidumont joined 14:27 domidumont joined
jnthn Alrighty, this is looking good, I think 14:31
timotimo i like to hear that 14:35
so the only thing remaining for using 8 bit per character storage is ... just doing it? :)
dalek arVM: b687df8 | jnthn++ | src/6model/reprs/MVMHash.h:
Remove a couple of now-unused macros.
 14:36
arVM: edd3bbf | jnthn++ | src/6model/reprs/MVMHash.h:
Remove unused macro parameter.

Remove the deprecated flattenropes op.
jnthn timotimo: We already do do it in various cases, I think? :)
But yeah, we could do it in more places :) 14:37
timotimo well, i mean ... store it like that forever :)
14:37 dalek joined
timotimo we used to blow it up for the benefit of hashes always 14:38
jnthn We never promote strings from 8-bit to 32-bit in-place now
timotimo awesome!
timotimo cracks fingers :P
jnthn And I'm about to remove the deprecated flattenropes o
*op
nwc10 you really have got it in for dalek today 14:39
14:40 ggoebel joined
jnthn :) 14:40
nwc10 I'm trying to remmeber - after this, how many known concurrency bugs? 14:41
jnthn Well, according to RT, 50ish :P 14:46
Though it's possible some of them boil down to what I just fixed
And that others share root causes
nwc10 Oh OK. How many known distinct causes
(Yes, what I was typing) 14:47
what next? well earned beer?
jnthn No, there's still more things needing attention :P
nwc10 are you sure that you have your priorities right here? :-)
jnthn Tomorrow is a national holiday, so plenty of time for beer then :) 14:48
nwc10 sounds terrible
jnthn Struggle for freedom and democracy day 14:49
Or, from the news I read this morning, struggling to find enough police to cope with the 30 different marches people have planned :P 14:50
brrt well, that is freedom, innit 14:52
jnthn 'tis fair enough, provided they don't make trouble, yes.
Hopefully the group that decided to fake an IS invasion of the old town square earlier this year will stay home :P 14:53
arVM: 9d5c874 | jnthn++ | src/strings/ops. (2 files):
Remove now-unused MVM_string_flatten function.
jnthn Well, that hopefully rounds off that task. 14:57
brrt does that mean that strings are now always flat? 15:06
jnthn brrt: No 15:08
It means that we don't have to make them flat in order to hash them 15:09
15:11 domidumont joined
timotimo brrt: we can re-use the mmapped string heap for a certain subset of our strings now! 15:12
well, we could have already done that before, i suppose 15:13
but now we won't need to immediately turn it into 32bit wide thingies as soon as they hit a hash
jnthn Um 15:17
Remember that compunits are GC-able
15:19 travis-ci joined
travis-ci MoarVM build failed. Jonathan Worthington 'Missing nullness checking.' 15:19
travis-ci.org/MoarVM/MoarVM/builds/176390873 github.com/MoarVM/MoarVM/compare/2...6e933a0688
15:19 travis-ci left 15:20 domidumont joined
jnthn wtf... 15:24
[Coke] looks like all mac failures. 15:26
I'll try a build.
jnthn May just be a timing issue 15:30
[Coke] Looking at the output, yah, maybe it got an nqp that didn't get the bump. 15:32
so far seems fine locally; also kicked off a rebuild of one of those failed subtasks. 15:33
timotimo jnthn: but don't the compunits hold the strings they own in a table?
jnthn: seems like if the compunit gets thrown out, i can go through the already-decoded strings and give them their own memory to hold their string data in 15:34
timotimo looks at the actual code rather than rely on memory and guesswork 15:38
dalek arVM: 20e968b | jnthn++ | src/io/ (2 files):
An nread of 0 is not an error.

And very occasionally happens. Just treat it as an empty buffer of incoming data.		 15:41
15:51 FROGGS joined
FROGGS o/ 15:51
jnthn o/ FROGGS 15:59
nwc10 if we all go o/ will the channel capsize? 16:10
(I think I asked this before, didn't I?)
jnthn It's the channel, not the boat on the channel :P
FROGGS actually lives quite near a channel 16:11
nwc10: btw, are you sad if I try to get all our Perl 5 scripts work on 5.8.8? 16:12
nwc10 I think I'm actually happy 16:13
because it seems to make sense to remove a small barrier to entry 16:14
FROGGS aye
nwc10 you could go for 5.8.5, although I'm told that the CentOS that ships that is about to fall off the end of the support conveyor belt
5.8.8 is the next logical one
FROGGS I see
it just happens that the AIX box I'm working with has 5.8.8 16:15
nwc10 5.8.8 is probably the best trade off
and xlc? :-)
FROGGS no :o)
gcc
brrt aha, okay
(late response, of course)
are strings going to be flat at all?
FROGGS xlc is discontinued it seems
nwc10 oh, boring 16:16
timotimo brrt: why not?
nwc10 it had all the best pedanticism
brrt or rather, how hard is string-character lookup going to be in the future
FROGGS nwc10: aix is still weird about linking...
nwc10 if your code worked on xlc, it had a fighting chance of working pretty much anywhere
timotimo we ought to always go through the grapheme iterator codepath
nwc10 (At least, on any OS still currently shipping)
the DEC compiler seems to have been awkward, but I never really got to meet it 16:17
brrt wonders how hard it is to support 5.8.8 for the tile template generator
my question is really, how hard would it be to specially JIT that codepath?
if you consider that loops are already kind of tricky
FROGGS brrt: we most likely dont hit that path I guess 16:18
brrt the grapheme iterator path i mean
FROGGS is talking about a platform with an ancient perl where we implement the jit
brrt aha :-) 16:23
my mistake for trying to engage two conversations at once
timotimo brrt: since strings are now properly read-only, we can have a fast path for flat strings up front, or alternatively just ask for a flattened string up front and then plough through a compact flat string in jitted code 16:25
brrt hmm 16:30
okay, that is pretty cool 16:31
16:33 domidumont joined
jnthn Note taht we can keep a spesh fact for "this string was flat" in regex code 16:33
Since we can spot uses of the indexingoptimized op 16:34
brrt hmm.... that would be decenlty useful
anyway, how far are we along with having branching guards.... 16:35
timotimo huh 16:38
i tried changing decoding latin1 to using MVM_grapheme_8 - though it probably should be ASCII instead if it doesn't have newlines in it? - but there's things that are beyond 127
171, 187, 160, and 162 16:39
m: say .&uniname for (160, 162, 171, 187)
camelia rakudo-moar 395f36: OUTPUT«NO-BREAK SPACE␤CENT SIGN␤LEFT-POINTING DOUBLE ANGLE QUOTATION MARK␤RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK␤»
timotimo i'm only surprised by the no-break space there
the cent sign is probably $¢
and the double angle quotation is also quite unsurprising
so i suppose that's fine. i'll put in a fallback to turn it into a 32bit grapheme storage thing when something's found that's outside 0..127 16:40
in theory, if there hasn't yet been a newline, we could use an ASCII stored string instead, since that allows us to use 0..255
actually, ASCII storage should also only allow 0..127 16:41
jnthn latin-1 goes up to 255 16:42
timotimo right, but the _8 storage is signed, so it's not useful for all of latin1 16:45
could not resolve lexical ꈤ翝
fantastic %)
ok, compiling rakudo now 16:46
16:51 travis-ci joined
travis-ci MoarVM build failed. Jonathan Worthington 'Missing nullness checking.' 16:51
travis-ci.org/MoarVM/MoarVM/builds/176390873 github.com/MoarVM/MoarVM/compare/2...6e933a0688
16:51 travis-ci left
timotimo Found /tmp/moar/bin/moar version 2016.10-69-gbc6e933, which is too old. 16:53
^- at least one of the mac builds failed like this
all mac builds failed like that 16:54
jnthn I think that the jobs were in the Travis queue for a bit, and then pulled NQP after it got a version bump 16:55
timotimo ah. could very well be 16:56
seems like a repl test is now hanging 17:00
oh, all the failing tests up until then were in perl5-integration 17:01
which could just be from Inline::Perl5 being too old or something 17:02
ah, i had a debug printf still in there
17:03 travis-ci joined
travis-ci MoarVM build failed. Jonathan Worthington 'Remove the deprecated flattenropes op.' 17:03
travis-ci.org/MoarVM/MoarVM/builds/176395412 github.com/MoarVM/MoarVM/compare/b...c68e5eb5d1
17:03 travis-ci left
timotimo but it still hangs :| 17:03
17:13 domidumont joined
dalek arVM/libatomic_ops-update: 6521f58 | FROGGS++ | 3rdparty/libatomic_ops/ (88 files):
remove executable flag where not needed
 18:24
arVM/libatomic_ops-update: 562c9dc | FROGGS++ | 3rdparty/libatomic_ops/configure.ac:
remove executable flag where not needed
 18:37
lizmat jnthn: could it be that there is a subtle difference in the way nqp::atpos works between shaped and unshaped arrays ? 18:45
aka, one is deconting and the other isnt? 18:46
dalek arVM/libatomic_ops-update: 6b7c1f7 | FROGGS++ | 3rdparty/libatomic_ops/src/atomic_ops_ (4 files):
add midding files
 18:51
lizmat jnthn: not so good news: HARNESS_TYPE=6 make spectest still not error free (segfault around S11) 19:00
gist.github.com/lizmat/916dd334cce...afbe537214 # problem with nqp::atpos / shaped arrays 19:03
afk to see some fantastic beasts& 19:04
19:41 domidumont joined
dalek Heuristic branch merge: pushed 97 commits to MoarVM/libatomic_ops-update by FROGGS 19:55
Heuristic branch merge: pushed 100 commits to MoarVM by FROGGS 20:06
FROGGS \m/
timotimo we gots a new libatomic_ops now? 20:07
FROGGS aye 20:10
timotimo cool 20:15
i wonder if we should grab newest libuv, too
dalek arVM/wip_aix: f7b1529 | FROGGS++ | / (6 files):
add all changes that are needed for aix

I will clean that up later, as well as splitting up this patch into smaller bits.		 20:23
FROGGS timotimo: since there are no issues with libuv from debian unstable and testing, I'd say yes
to all (probably 0) aix users here: that aix branch just works if you have libffi on your box 20:24
timotimo \o/ 20:26
timotimo teaches utf8 decoder to use 8 bit per char if possible 20:32
FROGGS gnight 20:34
stmuk_ watching interp.c compile on an old clang is about as interesting as watching paint dry 20:36
20:36 ggoebel joined
timotimo yeah, that's about right 20:36
boo!!! graphemes between 58 and 8745. 20:42
infix:sym<)>
o_O
m: say uniname(8745)
camelia rakudo-moar 58a482: OUTPUT«INTERSECTION␤»
timotimo i'm not sure i understand what's going on there 20:44
well, for one i'm not outputting valid utf8 to see what the thing is
so my console gets confused. fair enough
ok, these are our magical set operators and such 20:45
so "intersectin" was right all along %)
i mean ... we could still save some space if we introduce a 16bit storage category, but we might not want that 20:48
jnthn lizmat: Don't think it can be that simple; decont is a property of an op rather (stricly, the compilation of an op) rather than the REPR beneath it 21:15
22:09 travis-ci joined
travis-ci MoarVM build errored. Jonathan Worthington 'An nread of 0 is not an error. 22:09
travis-ci.org/MoarVM/MoarVM/builds/176412130 github.com/MoarVM/MoarVM/compare/9...e968be784b
22:09 travis-ci left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!