02:14 xtt joined
dalek arVM/smaller_string_storage: ec76e75 | timotimo++ | src/strings/latin1.c:
latin1 strings may be stored in 8bits sometimes

if their graphemes are strictly ASCII and the synthetics don't go out of range.		 02:25
arVM/smaller_string_storage: 4fd3220 | timotimo++ | src/strings/utf8.c:
some utf8-encoded strings also fit into 8bits.
timotimo looking into what the best way is to make indexingoptimized also able to have 8bit storage 02:27
(because if you can't sleep, might as well try to be productive)
02:48 ilbot3 joined
timotimo Unhandled exception: Error encoding UTF-8 string: could not encode codepoint 1918989427 03:14
uh oh, what have i done
geekosaur forgotten to handle your 8-bits somewhere 03:18
that's either "rats" or "star" 03:19
timotimo oh, you were kind enough to check the value as a char :)
that's cool
geekosaur well, did it to hex and I know most of the 20-7e from long experience :) 03:20
(i.e. staring at way too many hex dumps :p ) 03:21
timotimo ah
i forgot to turn the storage type into the other one when i do my conversion
i made a short function to do it so i don't have to retype the conversion over and over 03:22
and that function ended up buggy %)
i get through a rakudo build now :) 03:24
hm. with perl6 -e '' i get that function run 24 times 03:33
for the strings start, parse, ast, mast, mbc, moar ... over and over 03:34
that's from NQPHLL's my $compiler := HLL::Compiler.new() 03:36
the way it sets up its stages is by splitting a string by space 03:37
what i don't get is why it won't give me the "compression" output for cmdoptions ... perhaps nqp's compiler actually has constant folding for that?! 03:39
dalek arVM/smaller_string_storage: 79a161f | timotimo++ | src/strings/utf8.c:
kick out debug output for utf8 encoding
 04:16
arVM/smaller_string_storage: ddf58d7 | timotimo++ | src/strings/ops.c:
teach a few str functions to compress results into 8bit

collapse_strands, substring of complicated stranded strings, string_indexing_optimized, string_escape, string_flip, string_chr.
timotimo since it's spectest clean, i'm thinking of merging the branch 04:27
jesus, sharing the string memory buffer with the mmapped compunit is a noticable memory save 04:57
like half a megabyte for perl6 -e '' 04:58
05:05 mst joined
dalek arVM/share_string_mem_with_compunit: 7d31a88 | timotimo++ | src/ (4 files):
share some string's memory with compunit memory

if the string itself is already latin1, and it doesn't contain any crlf, then the memory will be 1:1 the same between what we decoded and what's in the compunit string heap blob.
we then make sure to un-share memory when the compunit dies.		 05:27
timotimo this doesn't seem quite as safe as the other parts in the branch, but it can probably go into master, too. perhaps after the release? 05:28
could be helpful to have an ASAN-enabled spec test run for this
timotimo tries to catch up on sleep missed tonight
05:30 xtt joined
timotimo 11407 strings got into the latin1 memory sharing thing for 104794 bytes in total, where the average size was 9.18 bytes 05:34
3 6 and 12 are the 25%, 50% and 75% percentiles 05:35
06:24 domidumont joined 06:57 domidumont joined 07:01 domidumont joined
nwc10 timotimo: SEGV paste.scsys.co.uk/539798 07:02
07:04 domidumont joined 07:15 domidumont joined 08:35 zakharyas joined 08:46 mtj_ joined 09:28 FROGGS joined 10:47 brrt joined
timotimo cool, thanks, nwc10 12:46
i can imagine what went wrong there, heh. 12:47
not sure why it didn't asploded on my end, though 12:50
once we have weak references implemented, strings that haven't been needed from the string heap can also be cleaned up 12:53
i'm imagining since we usually have a compilation stage and a run stage in our programs, that could be helpful
dalek arVM/share_string_mem_with_compunit: 530f3cf | timotimo++ | src/6model/reprs/MVMCompUnit.c:
prevent access to un-deserialized strings in gc_free
 12:56
timotimo spec tests are clean with that btw 13:29
lizmat still, it feels wise to merge this after the release 13:42
timotimo sure, no problem 13:43
lizmat many things aren't covered yet by spectests (as I've found out the hard way)
that's why I'm not going to commit anything important until after the release :-) 13:50
15:04 ggoebel joined
[Coke] this is why I don't like our single threaded nom-as-mainline-dev-and-release-branch strategy. Do wish we were taking more advantage of branching. 15:07
jnthn has had stuff in branches and merged it post-release before 15:08
timotimo right, we do that all the time
especially me, who is known for committing dangerous, broken, and plain low-grade code :s
jnthn I see it happen more in MoarVM than Rakudo, to be fair.
timotimo: You're smart enough to do it in a branch most of the time though :P :P 15:09
timotimo right
jnthn I still kinda like the automated delivery to release branch idea.
timotimo yeah, we ought to build that at some point 15:10
jnthn That is, we have a CD-style thing that every so often picks up the mainline development branch, stresstests it, tests a bunch of modules on it, and if all is well fast-forwards a release-candidate branch to match it.
And releases are cut from the latter. 15:11
lizmat fwiw, my experiences with branches isn't that good 15:16
not from a technical point of view, but from a code review / community point of view :-(
timotimo right, people end up not looking at it for months ... 15:17
the person who initiated the branch gets discouraged
jnthn lizmat: Doesn't stop you committing stuff you're working on in a branch just so you can keep working "as normal" on things whlie a release happens, and merging it after. 15:18
lizmat jnthn: git stash is my friend :-)
timotimo mine, too 15:19
jnthn Same, though I use it for different workflows than branches :)
timotimo right
i often use it to transplant changes i haven't committed yet across branches
because just checking stuff out, or "git pull" or something might complain 15:20
and especially git rebase. git rebase really doesn't want unstaged changes in your working copy
jnthn So, Rakudo release is this weekend?
lizmat I think that's the plan, afaik 15:21
jnthn OK, then I'll finish up my working week by doing the MoarVM release, so I don't have to worry about it (or block the Rakudo release) during the weekend. :)
lizmat jnthn++
jnthn Wow, I'd missed that 96d74e0c80bc6a happened 15:35
timotimo++
dalek arVM: 1e2d7b4 | jnthn++ | docs/ChangeLog:
Changelog entries for 2016.11.
 15:41
arVM: 21193c5 | jnthn++ | VERSION:
Bump VERSION.
 15:42
timotimo oh, that 15:47
yeah, that was cool :)
dalek arVM: 81fd0ac | jnthn++ | docs/ChangeLog:
Missed a Changelog entry.
 15:48
jnthn timotimo: What does it actually mean, though? Can valgrind catch overruns *within* FSA-allocated things?
Or better put: can it catch overruns of memory allocated using the FSA? 15:49
timotimo yes
i've added support for a redzone
that is exactly that
jnthn Nice
timotimo one thing we're missing is a long queue for freeing stuff to more agressively trigger use-after-free 15:50
i.e. keep 10k allocated bits around before actually starting to free stuff
jnthn I normally flipped the FSA into the debug mode (just uses malloc and records the size for verification)
To use valgrind
:)
Well, when I was bug-hutning and suspected such overruns, that is :) 15:51
timotimo makes sense
but when you're trying to change things in the FSA, basically turning the FSA off isn't the way to go ;)
jnthn Indeed 15:52
I was more thinking debugging things that (mis-)use the FSA
OK, anybody got any reason for me not to pull the release trigger? 15:53
Feel free to glance the Changelog for mistakes...
timotimo right
timotimo looks over the changelog 15:54
the stuff in it seems good. haven't looked if anything's missing 15:56
you generally don't miss anything because of the way you build the changelog, so that's not something i worry about
jnthn I did it using git log as usual, but this time there were a ton of libatomicops commits imported
Condensed those to one entry :)
timotimo right 15:57
jnthn Full build of the release = 18s 15:59
In my VM with 4 cores allocated to it
Not bad :)
timotimo \o/
clearly that's not clang
jnthn gcc :) 16:00
www.moarvm.org/releases/MoarVM-2016.11.tar.gz 16:02
moritz jnthn++ 16:03
timotimo \o/ 16:04
should i recklessly merge my two branches, since they are both spectest-clean? ;)
jnthn I dunno, are they good? :P 16:05
timotimo i think they are good
good for memory usage :)
moritz then go for it
timotimo yay! 16:06
16:14 travis-ci joined
travis-ci MoarVM build failed. Jonathan Worthington 'Changelog entries for 2016.11.' 16:14
travis-ci.org/MoarVM/MoarVM/builds/177055968 github.com/MoarVM/MoarVM/compare/1...2d7b42bd17
16:14 travis-ci left
[Coke] ohnoes. 16:16
timotimo oh no, the change log blew it all up! 16:17
17:16 FROGGS joined
FROGGS o/ 17:16
timotimo o/ 17:20
17:25 domidumont joined
jnthn FROGGS: Don't suppose you've got a list of the platforms you've done MoarVM portability improvements for this month? Thought it'd be nice to put in the release blurb on the homepage for this month... :) 18:11
FROGGS jnthn: do you have open office? 18:12
(or libre office)
jnthn Surely one of them 18:14
Libre
FROGGS froggs.de/perl6/platform_matrix.html 18:16
pass means nqp and rakudo pass their 'make test'
ffi means that libffi is supported on that platform
grey means, there is no such os/arch combination 18:17
the qemu column means that there is a qemu system emulator available
jnthn: does that help? 18:18
[Coke] this page is in maltese!
FROGGS [Coke]: the thing I just posted? O.o 18:19
psch guesses chrome offering to translate
geekosaur chrome here didn't see a problem 18:20
FROGGS jnthn: I can give you a list of operating systems that needed help too
but one can read that in the commits too
[Coke] psch: ayup 18:21
FROGGS: ayup
japhb FROGGS: Is it your intent that any os/arch that has *both* qemu and ffi support should be supportable by MoarVM? 18:22
FROGGS japhb: qemu support doesnt matter, but makes it easier for me 18:25
japhb: every os that is supported by libuv, libffi, libatomic_ops etc and which has a modern enough arch with enough ram is a valid target
and I try to get rakudo there if I can 18:26
example: m68k is most likely not a desirable target 18:30
japhb FROGGS: A laudable goal ... do you have support lists for libuv and libatomic_ops as well? Or are they just "try them and see if they compile and pass tests"?
jnthn FROGGS: Yes, thanks :) 18:32
What does "ffi" in a column mean? That ffi support is the blocker? 18:33
lizmat jnthn: is there a reason why nqp::split is generating a list_o rather than a list_s ? 18:34
timotimo maybe it means we have to use libffi instead of dyncall? 18:35
jnthn lizmat: To save another loop, I guess
lizmat another loop ?
timotimo did y'all know our index and rindex ops don't actually do boyer-moore? 18:36
jnthn lizmat: In most cases we need to hand back boxed strings; it'd take another iteration to do that.
lizmat ok 18:37
jnthn I'm not sure whether the Perl 6 split actually uses nqp::split though? 18:38
Since it can potentially work lazily
timotimo right, probably uses index instead
lizmat split( Str(Cool) ) uses nqp::split 18:40
and returns a fully reified list 18:41
jnthn Ah, OK
That's...probably often faster
m: say 'omg'.split(/m/).WHAT
camelia rakudo-moar b5aa3c: OUTPUTĀ«(List)ā¤Ā»
jnthn m: say 'omg'.split('m').WHAT
lizmat well, if the string is 2G, then probably not
camelia rakudo-moar b5aa3c: OUTPUTĀ«(List)ā¤Ā»
jnthn Good, it's consistent
True :) 18:42
jnthn is a tiny bit surprised that we didn't return a Seq there
m: say 'omg'.comb(/m/).WHAT
camelia rakudo-moar b5aa3c: OUTPUTĀ«(List)ā¤Ā»
jnthn Ah, consistent with that also :)
(if the string is 2G) depends how many pieces it breaks up in to... :) 18:43
bbi30 18:44
FROGGS jnthn: ffi means that libffi is available according to the libffi.org page 19:03
jnthn: though, that does not mean that the other deps, namely libuv, are available
jnthn: I either fixed compilation or fixed issues in tests for linux on: arm, aarch64(arm64), mips, mips64, ppc, ppc64, s390x, x32 and for solaris@x86_64, as well as making moarvm and nqp build on aix@ppc 19:11
japhb: there is github.com/libuv/libuv/issues/983 19:13
japhb: and libatomic_ops runs almost everywhere I think, that includes hpux, true64, irix, ... 19:16
jnthn: ahh, btw, the aix changes are still in a branch 19:49
19:50 domidumont joined
dalek href="https://moarvm.org:">moarvm.org: c87d9cc | jnthn++ | / (2 files):
Update website for 2016.11 release.
 20:27
jnthn Our 35th release. 20:28
Time flies... 20:29
FROGGS wow 20:38
looks like I can get nqp on aix test clean quite easily 21:31
the weird test failures I had yesterday were about a too recent libuv
jnthn Nice!
FROGGS it only fails two tests, and these two were about inf/neginf... I hacked something up quickly so that moar builds 21:32
and I just need to fix that
Stage parse : MoarVM panic: Memory allocation failed; could not allocate 16384 bytes 21:47
gmake: *** [CORE.setting.moarvm] Error 1
:~(
22:59 MrJones joined
MrJones hi 23:00
can moarvm be used as a static library out of a C program which then launches it on some runtime-provided bytecode blob?

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!