dalek arVM/smaller_string_storage: ec76e75 | timotimo++ | src/strings/latin1.c:
latin1 strings may be stored in 8bits sometimes

if their graphemes are strictly ASCII and the synthetics don't go out of range.		 02:25
arVM/smaller_string_storage: 4fd3220 | timotimo++ | src/strings/utf8.c:
some utf8-encoded strings also fit into 8bits.
timotimo looking into what the best way is to make indexingoptimized also able to have 8bit storage 02:27
(because if you can't sleep, might as well try to be productive)
timotimo Unhandled exception: Error encoding UTF-8 string: could not encode codepoint 1918989427 03:14
uh oh, what have i done
geekosaur forgotten to handle your 8-bits somewhere 03:18
that's either "rats" or "star" 03:19
timotimo oh, you were kind enough to check the value as a char :)
that's cool
geekosaur well, did it to hex and I know most of the 20-7e from long experience :) 03:20
(i.e. staring at way too many hex dumps :p ) 03:21
timotimo ah
i forgot to turn the storage type into the other one when i do my conversion
i made a short function to do it so i don't have to retype the conversion over and over 03:22
and that function ended up buggy %)
i get through a rakudo build now :) 03:24
hm. with perl6 -e '' i get that function run 24 times 03:33
for the strings start, parse, ast, mast, mbc, moar ... over and over 03:34
that's from NQPHLL's my $compiler := HLL::Compiler.new() 03:36
the way it sets up its stages is by splitting a string by space 03:37
what i don't get is why it won't give me the "compression" output for cmdoptions ... perhaps nqp's compiler actually has constant folding for that?! 03:39
dalek arVM/smaller_string_storage: 79a161f | timotimo++ | src/strings/utf8.c:
kick out debug output for utf8 encoding
 04:16
arVM/smaller_string_storage: ddf58d7 | timotimo++ | src/strings/ops.c:
teach a few str functions to compress results into 8bit

collapse_strands, substring of complicated stranded strings, string_indexing_optimized, string_escape, string_flip, string_chr.
timotimo since it's spectest clean, i'm thinking of merging the branch 04:27
jesus, sharing the string memory buffer with the mmapped compunit is a noticable memory save 04:57
like half a megabyte for perl6 -e '' 04:58
dalek arVM/share_string_mem_with_compunit: 7d31a88 | timotimo++ | src/ (4 files):
share some string's memory with compunit memory

if the string itself is already latin1, and it doesn't contain any crlf, then the memory will be 1:1 the same between what we decoded and what's in the compunit string heap blob.
we then make sure to un-share memory when the compunit dies.		 05:27
timotimo this doesn't seem quite as safe as the other parts in the branch, but it can probably go into master, too. perhaps after the release? 05:28
could be helpful to have an ASAN-enabled spec test run for this
timotimo tries to catch up on sleep missed tonight
timotimo 11407 strings got into the latin1 memory sharing thing for 104794 bytes in total, where the average size was 9.18 bytes 05:34
3 6 and 12 are the 25%, 50% and 75% percentiles 05:35
nwc10 timotimo: SEGV paste.scsys.co.uk/539798 07:02
timotimo cool, thanks, nwc10 12:46
i can imagine what went wrong there, heh. 12:47
not sure why it didn't asploded on my end, though 12:50
once we have weak references implemented, strings that haven't been needed from the string heap can also be cleaned up 12:53
i'm imagining since we usually have a compilation stage and a run stage in our programs, that could be helpful
dalek arVM/share_string_mem_with_compunit: 530f3cf | timotimo++ | src/6model/reprs/MVMCompUnit.c:
prevent access to un-deserialized strings in gc_free
 12:56
timotimo spec tests are clean with that btw 13:29
lizmat still, it feels wise to merge this after the release 13:42
timotimo sure, no problem 13:43
lizmat many things aren't covered yet by spectests (as I've found out the hard way)
that's why I'm not going to commit anything important until after the release :-) 13:50
[Coke] this is why I don't like our single threaded nom-as-mainline-dev-and-release-branch strategy. Do wish we were taking more advantage of branching. 15:07
jnthn has had stuff in branches and merged it post-release before 15:08
timotimo right, we do that all the time
especially me, who is known for committing dangerous, broken, and plain low-grade code :s
jnthn I see it happen more in MoarVM than Rakudo, to be fair.
timotimo: You're smart enough to do it in a branch most of the time though :P :P 15:09
timotimo right
jnthn I still kinda like the automated delivery to release branch idea.
timotimo yeah, we ought to build that at some point 15:10
jnthn That is, we have a CD-style thing that every so often picks up the mainline development branch, stresstests it, tests a bunch of modules on it, and if all is well fast-forwards a release-candidate branch to match it.
And releases are cut from the latter. 15:11
lizmat fwiw, my experiences with branches isn't that good 15:16
not from a technical point of view, but from a code review / community point of view :-(
timotimo right, people end up not looking at it for months ... 15:17
the person who initiated the branch gets discouraged
jnthn lizmat: Doesn't stop you committing stuff you're working on in a branch just so you can keep working "as normal" on things whlie a release happens, and merging it after. 15:18
lizmat jnthn: git stash is my friend :-)
timotimo mine, too 15:19
jnthn Same, though I use it for different workflows than branches :)
timotimo right
i often use it to transplant changes i haven't committed yet across branches
because just checking stuff out, or "git pull" or something might complain 15:20
and especially git rebase. git rebase really doesn't want unstaged changes in your working copy
jnthn So, Rakudo release is this weekend?
lizmat I think that's the plan, afaik 15:21
jnthn OK, then I'll finish up my working week by doing the MoarVM release, so I don't have to worry about it (or block the Rakudo release) during the weekend. :)
lizmat jnthn++
jnthn Wow, I'd missed that 96d74e0c80bc6a happened 15:35
timotimo++
dalek arVM: 1e2d7b4 | jnthn++ | docs/ChangeLog:
Changelog entries for 2016.11.
 15:41
arVM: 21193c5 | jnthn++ | VERSION:
Bump VERSION.
 15:42
timotimo oh, that 15:47
yeah, that was cool :)
dalek arVM: 81fd0ac | jnthn++ | docs/ChangeLog:
Missed a Changelog entry.
 15:48
jnthn timotimo: What does it actually mean, though? Can valgrind catch overruns *within* FSA-allocated things?
Or better put: can it catch overruns of memory allocated using the FSA? 15:49
timotimo yes
i've added support for a redzone
that is exactly that
jnthn Nice
timotimo one thing we're missing is a long queue for freeing stuff to more agressively trigger use-after-free 15:50
i.e. keep 10k allocated bits around before actually starting to free stuff
jnthn I normally flipped the FSA into the debug mode (just uses malloc and records the size for verification)
To use valgrind
:)
Well, when I was bug-hutning and suspected such overruns, that is :) 15:51
timotimo makes sense
but when you're trying to change things in the FSA, basically turning the FSA off isn't the way to go ;)
jnthn Indeed 15:52
I was more thinking debugging things that (mis-)use the FSA
OK, anybody got any reason for me not to pull the release trigger? 15:53
Feel free to glance the Changelog for mistakes...
timotimo right
timotimo looks over the changelog 15:54
the stuff in it seems good. haven't looked if anything's missing 15:56
you generally don't miss anything because of the way you build the changelog, so that's not something i worry about
jnthn I did it using git log as usual, but this time there were a ton of libatomicops commits imported
Condensed those to one entry :)
timotimo right 15:57
jnthn Full build of the release = 18s 15:59
In my VM with 4 cores allocated to it
Not bad :)
timotimo \o/
clearly that's not clang
jnthn gcc :) 16:00
www.moarvm.org/releases/MoarVM-2016.11.tar.gz 16:02
moritz jnthn++ 16:03
timotimo \o/ 16:04
should i recklessly merge my two branches, since they are both spectest-clean? ;)
jnthn I dunno, are they good? :P 16:05
timotimo i think they are good
good for memory usage :)
moritz then go for it
timotimo yay! 16:06
travis-ci MoarVM build failed. Jonathan Worthington 'Changelog entries for 2016.11.' 16:14
travis-ci.org/MoarVM/MoarVM/builds/177055968 github.com/MoarVM/MoarVM/compare/1...2d7b42bd17
[Coke] ohnoes. 16:16
timotimo oh no, the change log blew it all up! 16:17
FROGGS o/ 17:16
timotimo o/ 17:20
jnthn FROGGS: Don't suppose you've got a list of the platforms you've done MoarVM portability improvements for this month? Thought it'd be nice to put in the release blurb on the homepage for this month... :) 18:11
FROGGS jnthn: do you have open office? 18:12
(or libre office)
jnthn Surely one of them 18:14
Libre
FROGGS froggs.de/perl6/platform_matrix.html 18:16
pass means nqp and rakudo pass their 'make test'
ffi means that libffi is supported on that platform
grey means, there is no such os/arch combination 18:17
the qemu column means that there is a qemu system emulator available
jnthn: does that help? 18:18
[Coke] this page is in maltese!
FROGGS [Coke]: the thing I just posted? O.o 18:19
psch guesses chrome offering to translate
geekosaur chrome here didn't see a problem 18:20
FROGGS jnthn: I can give you a list of operating systems that needed help too
but one can read that in the commits too
[Coke] psch: ayup 18:21
FROGGS: ayup
japhb FROGGS: Is it your intent that any os/arch that has *both* qemu and ffi support should be supportable by MoarVM? 18:22
FROGGS japhb: qemu support doesnt matter, but makes it easier for me 18:25
japhb: every os that is supported by libuv, libffi, libatomic_ops etc and which has a modern enough arch with enough ram is a valid target
and I try to get rakudo there if I can 18:26
example: m68k is most likely not a desirable target 18:30
japhb FROGGS: A laudable goal ... do you have support lists for libuv and libatomic_ops as well? Or are they just "try them and see if they compile and pass tests"?
jnthn FROGGS: Yes, thanks :) 18:32
What does "ffi" in a column mean? That ffi support is the blocker? 18:33
lizmat jnthn: is there a reason why nqp::split is generating a list_o rather than a list_s ? 18:34
timotimo maybe it means we have to use libffi instead of dyncall? 18:35
jnthn lizmat: To save another loop, I guess
lizmat another loop ?
timotimo did y'all know our index and rindex ops don't actually do boyer-moore? 18:36
jnthn lizmat: In most cases we need to hand back boxed strings; it'd take another iteration to do that.
lizmat ok 18:37
jnthn I'm not sure whether the Perl 6 split actually uses nqp::split though? 18:38
Since it can potentially work lazily
timotimo right, probably uses index instead
lizmat split( Str(Cool) ) uses nqp::split 18:40
and returns a fully reified list 18:41
jnthn Ah, OK
That's...probably often faster
m: say 'omg'.split(/m/).WHAT
camelia rakudo-moar b5aa3c: OUTPUT«(List)␤»
jnthn m: say 'omg'.split('m').WHAT
lizmat well, if the string is 2G, then probably not
camelia rakudo-moar b5aa3c: OUTPUT«(List)␤»
jnthn Good, it's consistent
True :) 18:42
jnthn is a tiny bit surprised that we didn't return a Seq there
m: say 'omg'.comb(/m/).WHAT
camelia rakudo-moar b5aa3c: OUTPUT«(List)␤»
jnthn Ah, consistent with that also :)
(if the string is 2G) depends how many pieces it breaks up in to... :) 18:43
bbi30 18:44
FROGGS jnthn: ffi means that libffi is available according to the libffi.org page 19:03
jnthn: though, that does not mean that the other deps, namely libuv, are available
jnthn: I either fixed compilation or fixed issues in tests for linux on: arm, aarch64(arm64), mips, mips64, ppc, ppc64, s390x, x32 and for solaris@x86_64, as well as making moarvm and nqp build on aix@ppc 19:11
japhb: there is github.com/libuv/libuv/issues/983 19:13
japhb: and libatomic_ops runs almost everywhere I think, that includes hpux, true64, irix, ... 19:16
jnthn: ahh, btw, the aix changes are still in a branch 19:49
dalek href="https://moarvm.org:">moarvm.org: c87d9cc | jnthn++ | / (2 files):
Update website for 2016.11 release.
 20:27
jnthn Our 35th release. 20:28
Time flies... 20:29
FROGGS wow 20:38
looks like I can get nqp on aix test clean quite easily 21:31
the weird test failures I had yesterday were about a too recent libuv
jnthn Nice!
FROGGS it only fails two tests, and these two were about inf/neginf... I hacked something up quickly so that moar builds 21:32
and I just need to fix that
Stage parse : MoarVM panic: Memory allocation failed; could not allocate 16384 bytes 21:47
gmake: *** [CORE.setting.moarvm] Error 1
:~(
MrJones hi 23:00
can moarvm be used as a static library out of a C program which then launches it on some runtime-provided bytecode blob?

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!