01:49 ilbot3 joined
Geth_ MoarVM: 916403fd76 | (Samantha McVey)++ | src/strings/ops.c
Start off collapsing strands as 8bit instead of 32bit

Here instead of starting as 32-bit and then at the end converting the already iterated 32-bit blob to 8bit, we instead start off converting as 8-bit and as soon as we see a character that won't fit, we do the following:
... (12 more lines)		 03:03
MoarVM: 42f7fb2d18 | (Samantha McVey)++ (committed using GitHub Web editor) | src/strings/ops.c
Merge pull request #616 from samcv/collapse-strands-bitwork

Start off collapsing strands as 8bit instead of 32bit
MoarVM: a739fc829f | (Samantha McVey)++ | src/strings/ops.c
Don't trigger re_nfg on concat for base character + combiner

This loosens the requirements so that only last_a needs to have a Canonical Combining Class of 0.		 04:59
05:48 brrt joined 06:32 domidumont joined 06:38 domidumont joined 06:43 brrt joined 06:51 lizmat joined
Geth_ MoarVM/even-moar-jit: 374beb9ade | (Bart Wiegmans)++ | docs/jit/todo.org
Document win64 presumed call arg failure

Not entirely obvious why it happens, but there is certainly something strange going on.		 07:11
07:58 zakharyas joined 08:01 domidumont joined
timotimo github.com/MoarVM/MoarVM/compare/r...s?expand=1 - why didn't we actually need this? 08:48
08:56 brrt joined
jnthn morning o/ 08:57
brrt ohai, jnthn, timotimo
timotimo yo 08:58
okay so we have a bug now where someone is running a long-running script and updates rakudo, then moarvm explodes when new code is touched
clearly it's accidentally lazy-reading new bytecode to do some code it hadn't run before 08:59
we might want to have rakudo put the core setting .moarvm file in a versioned place
jnthn Yeah, that sounds like "mmap'd file being changed under us" or some such 09:00
timotimo alternatively, is there a way to open the file such that any modifications will be invisible to us?
yup
jnthn Maybe
timotimo reads man page 09:01
MAP_PRIVATE
Create a private copy-on-write mapping.
nwc10 MAP_PRIVATE - ... It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region.
yay POSIX.
jnthn Or can we install in a smarter way?
timotimo It
nwc10 no, we can't be sure.
timotimo is unspecified whether changes made to the file after the
mmap() call are visible in the mapped region.
;(
jnthn Such that the filename points to a different inode or whatever?
nwc10 that plan seems better 09:02
jnthn I kinda thought that we'd already have that...
But if we're seeing this issue it must be actually opening the existing file, truncating, and writing to it
timotimo oh, you mean it should mv the file away, then create a new one 09:03
that ought to work
or maybe even just unlink 09:04
that'll still get us into a spot of bother if an already-running script suddenly decides to "use" some new code that's installed in the same way
clearly the solution should be to prepend new versions to the file and when you see an unsupported moar version being required you skip ahead to the next bit
unbounded growth for our .moarvm files \o/ 09:05
jnthn :P
"clearly" :P
As to string scanning/flattening, I think "only the regex engine flattens things out" is an easily learnable guideline 09:06
timotimo i'll send a part of our findings off to that bug report 09:10
or maybe write a fix for the installation part of the makefile 09:11
so we're currently installing it via cp 09:13
it has a --remove-destination flag - at least on my system
maybe i'll just put an RM_F in front 09:17
jnthn Hmm, curious 09:18
For
CORE.setting compilation, the amount of time the spesh worker spends is:
Total statistics time: 63.479ms 09:19
Total planning time: 4.184ms
Total specialization time: 3182.48ms
Geth_ MoarVM/spesh-worker: 76f127995a | (Jonathan Worthington)++ | tools/spesh-time.p6
Add a script to sum up logged spesh times.
 09:20
jnthn So there goes the theory that the compilation is so slow 'cus spesh is doing an insane amount of work... 09:21
Maybe it really is just that we're missing incorporating various bits of the logging data still :) 09:22
Useful new script to have, anyway ;)
timotimo so gnu make offers a function called addprefix which would let me generate the to-delete paths from an array like M_PERL6_LANG_OUTPUT 09:24
i can't find anything about portability questions
i'll write a perl5 snippet instead i guess? 09:28
actually i can already run perl6 at this point
jnthn Yeah 09:29
nine jnthn: btw what do you think about the suggestion to ship a make.exe with MoarVM sources, so we'd no longer be held back by nmake? 09:30
Geth_ MoarVM/spesh-worker: 2f6f6868df | (Jonathan Worthington)++ | 3 files
Pass spesh plan into optimize phase.
jnthn nine: Not really too convinced by it 09:31
timotimo should i use rm-f on the output of the script or should i .IO.unlink inside perl6?
nwc10 Likely then any tarballs would get impounded by (some) virus scanners 09:32
jnthn I suspect there are other make variants out there that are also not smart
nwc10 certainly HP-UX make
er, HP/UX 09:33
jnthn And yeah, shipping binary blobs in general is kinda icky
timotimo is HP/UX the ibm thing that's still getting development for it funded? 09:34
no, HP isn't ibm
what was the one i'm thinking about ..
jnthn I also suspsect that anything really clever you might try to do with makefiles would be easier to understand if it were done by the makefile calling a script 09:35
timotimo ok so how do i put a $_ in between '' inside a makefile without make replacing it with "" 09:37
just introduce a $_ variable that has the contents $_?
09:39 brrt joined
nine jnthn: well "clever" is a very subjective thing with dmake being even too stupid for simple backticks - which makes even calling the clever script impossible 09:40
timotimo ha, i can just .Str instead of $_
jnthn We could just write boring, simple, makefiles. :) 09:42
It's apparently got us this far, so... :) 09:43
09:46 robertle joined
Geth_ MoarVM/spesh-worker: a7e6d46595 | (Jonathan Worthington)++ | src/spesh/optimize.c
Re-instate lexical per invocant type optimization.
 10:12
jnthn Putting that back shaved a good chunk off CORE.setting compilation 10:13
Remaining to re-instate are logged facts, size-based thresholds, and OSR 10:16
After that I'll probably do a round of tuning and bug-fixing to try and get this mergeable, before going on to the more clever things we should be able to do with it
nine exciting times :) 10:22
jnthn Putting the logged facts back is...interesting :) 10:32
(We do the logging in a totally different way now) 10:33
Geth_ MoarVM/spesh-worker: 33c8635db4 | (Jonathan Worthington)++ | 2 files
Mark further ops with :logged.

So we have the index under which the logged values can be located.		 10:44
timotimo more opportunities for a cross-reference tool ... 10:48
10:51 brrt joined 10:58 brrt joined 10:59 vendethiel joined
nwc10 Stage parse : 612.736 11:06
jnthn: master is just under 500 with ASAN 11:07
jnthn Seems you get the same size of improvement since yesterday as I got, then :) 11:08
Well, ratio of :) 11:09
lunch; making good progress on adding back the other logged facts though 11:10
12:19 zakharyas joined 12:21 brrt joined 12:22 lucasb joined
Geth_ MoarVM/spesh-worker: 24f3f2fb86 | (Jonathan Worthington)++ | 8 files
Rework guard ops.

The container-related guards are now gone; we'll guard at the point of deopt instead. There's a new guard op which just guards on STable, not on concrete vs. type object.		 13:04
jnthn bah, typo
Geth_ MoarVM/spesh-worker: 8a50ac77a8 | (Jonathan Worthington)++ | 8 files
Rework guard ops.

The container-related guards are now gone; we'll guard at the point of decont instead. There's a new guard op which just guards on STable, not on concrete vs. type object.
jnthn Better :)
Now getting a pretty obscure crash after putting the log-based facts/guards back in 13:11
Of course, it would just have to happen in a 200 basic block frame :/ 13:30
bah, found it...mis-inserted the guard 13:47
Now NQP build/test is happy again
Hm, Rakudo one trips over a real NULL sneaking into an argument list 13:52
Geth_ MoarVM/spesh-worker: 5ca9e628f8 | (Jonathan Worthington)++ | 2 files
Add facts/guards based on logs.
 13:55
jnthn Latest problem is another issue in another giant frame 14:03
This time `unwanted` from Perl6::Actions 14:04
nwc10 ASAN goes Boom on the NULL pointer 14:13
just in red. no technicolour barfage :-( 14:14
Geth_ MoarVM/spesh-worker: 5068ef91dd | (Jonathan Worthington)++ | 5 files
Re-introduce thesholds based on bytecode size.
 14:26
nwc10 does that fix it for real, or just work around a problem?
jnthn Doesn't fix it for real, but does re-apply the same kind of hiding that we had in master :) 14:28
It's entirely possible (will need to check) that the thresholding was all that kept us from stumbling on this in master
14:28 domidumont1 joined
jnthn The spesh "torture test" that should show these up is MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 14:31
And that does show up some problems 14:32
timotimo very thunderstorm right now 14:52
jnthn Same here :D 14:53
Most welcome
timotimo aye 14:54
15:07 brrt joined
Geth_ MoarVM/spesh-worker: aaafe12f73 | (Jonathan Worthington)++ | 3 files
Clean up spesh stats no longer updated.
 15:09
jnthn That should bring memory down some :)
15:09 geekosaur joined
jnthn Darn, spectset looks awful :/ 15:10
*spectest
But at least I managed to find it doing something wrong in a frame wiht just a handful of basic blocks 15:23
15:29 domidumont joined 15:31 AlexDaniel joined 15:42 hoelzro joined
jnthn Well, don't think I'll find it today; too tired now 16:01
Hopefully tomorrow
16:08 zakharyas joined 16:15 domidumont joined 16:33 lizmat joined 17:06 robertle joined 17:22 Util joined
Geth_ MoarVM: 499059dccf | (Jonathan Worthington)++ | 3 files
Enable native callbacks on different threads.

Prior to this change, the callback was tied to the thread that passed it to the native library, so any attempt to call it on another thread would crash and burn. This issue can be resolved by conveying the instance, and resolving it to a thread context by using the native thread ID.		 19:16
jnthn That could be a nasty one 19:17
Ended up with two threads trying to use the same thread context. You get some very, very interesting failure modes :P
timotimo oh, huh. interesting.
timotimo BBL, groceries and such 19:21
nwc10 jnthn: $ ./nqp t/nqp/066-pararole.t 19:41
MoarVM panic: Spesh arg guard: trying to add duplicate result for same guard
nqp master, MoarVM origin/spesh-worker
and ./perl6-m t/04-nativecall/21-callback-other-thread.t goes SEGV 20:06
jnthn On spesh-worker? 20:09
Yeah, that's what happened before the fix I put in above :)
The spesh arg guard dupe is odd, though you can get it to happen in NQP build with setting MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 20:10
I'm guessing we must somehow end up with a duplicate in the plan, but I dunno how that could happen
nwc10 yes 20:13
yes on spesh-worker
jnthn Guess I'll be doing some more bug hunting tomorrow :) 20:15
samcv jnthn, i'm going to make a grapheme codepoint iterator function ok 21:05
MVM_grapheme_ci_init MVM_grapheme_ci_has_more MVM_grapheme_ci_get_grapheme
can deduplicate some code that does it and make it easier to iterate on a synthetic 21:06
jnthn samcv: Sounds sensible enough 21:30
Slightly breaks the MVM_subsystemname_foo naming scheme I guess :)
samcv hmm 21:34
well
MVM_string_grapheme_foo
whatever you want to call it i guess 21:35
well my code seems to work. pretty useful 21:36
gist.github.com/94faeab2ea8a1a3e1b...d2189d3909 21:37
i may optimize it a little more and have MVM_grapheme_ci_has_more check ci->visited_synth_codes == ci->total_synth_codes instead hm 21:38
anyway it works
jnthn "Can only be used on synthetics" 21:44
It looks like it should work fine on non-synthetics too? :)
samcv well it can heh
i wrote that before i added a non-synthetic case
these functions make me very happy 21:45
jnthn Yeah, it's good that it handles the non-synthetic case
Avoids accidents :)
+1 to adding them, maybe with a name tweak 21:46
samcv though i might tweak it to not return the base character and just return the total number of cp i nthe synthetic 21:47
hm
not sure how i'd do that but hm
jnthn It'd match the APIs of the others if init didn't return anything I guess
But maybe the discrepancy is convenient? 21:48
21:50 lizmat joined
samcv i could set visited synth codes to 0 when i init it maybe 21:51
and then return the base character that way
oh i see why. the base char isn't stored in the codepoint iterator 21:52
that's why i return it
jnthn Guess it hinges on if things using the codepoint interator will generally want the base char separately or not
If they all just want the codepoints one at a time then they'll all have to work around the first one being delivered differently. 21:53
samcv so i'd need to store the base char in the ci
hm 21:54
so yeah i could add a basechar storage point to the ci
and add that to the ci struct 21:55
would that be ok jnthn?
jnthn Sounds fine to me. 21:56
samcv ok cool
jnthn Provided the use cases on the whole prefer it that way :)
Plus the consistency with the other iterators is nice
samcv yeah
i think the consistency might be worth doing it that way
jnthn, can i have the MVM_string_graphem_ci_init return the number of codepoints in the iterator though? 21:57
the other one is void. but getting the total length would be quite useful
jnthn samcv: Yeah, I think that'd be OK 22:00
samcv unlike iterating on a string, we can know right then how long it is 22:01
jnthn Aye 22:03
samcv almost done getting synthetics to work in the concat thing :) 22:54
jnthn, MVMROOT doesn't like me. not sure what i'm doing wrong. but it's giving me errors about suplynig it four arguments 23:13
jnthn Oh, the joy of C macros :) 23:15
The C preprocessor can get confused if you do a variable declaration 23:16
Like
MVMint32 i, n;
It somehow thinks the , in there is a macro argument separator?!
samcv yeah i have a var declaration above the MVMROOT
jnthn No, this is just inside of it
samcv gist.github.com/5e39280fe1639c0258...2a5a5547e2 23:17
jnthn MVMCodepointIter last_a_ci, first_b_ci;
Try breaking that into two separate decls
(Yes, this is horrible. :/)
samcv yay :) 23:20
timotimo wow, that's pretty incredible 23:32
samcv commas not allowed 23:39
or something. hah.
timotimo no comma allowed 23:40
like the topic of the python channel: NO ,
23:46 AlexDaniel joined
Geth_ MoarVM/adjust_nursery_stringops: bb7e757265 | (Timo Paulssen)++ | src/strings/ops.c
shrink remaining nursery when huge strings get made
 23:51
timotimo i thought i had already pushed tha
Geth_ MoarVM: cb98db0fdd | (Samantha McVey)++ | src/strings/iter.h
Add functions for iterating the codepoints of a grapheme

These functions act similarly to MVM_string_ci functions except MVM_string_grapheme_ci_init returns the number of codepoints in the grapheme instead of void. If using these you should exclusively use the associated MVM_string_grapheme_ci functions.		 23:55
MoarVM: 092cc0d9f2 | (Samantha McVey)++ | src/strings/ops.c
Don't re_nfg when last_a or first_b are synthetics in concat

This also removes the canonical combining class restriction as well. If both codepoints aren't synthetic it will construct the renormalized_section from that. Otherwise it will use the new MVM_string_grapheme_ci functions to extract the codepoints from the graphemes.
samcv yay
jnthn samcv++ timotimo++ 23:59
Righty, cool enough to sleep. 'night o/

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!