timotimo gist.github.com/timo/f85f49d0e4b1e...e64caeb3cb - anyway, here's what i got for the offset setting and usage in the arg guard tree interpreter thingie. something's wrong, but i don't quite know what. 01:20
actually, let me change that back to the version that would complain if the two methods disagreed
'k, bedtime. 01:21
01:48 ilbot3 joined 02:31 TimToady joined 03:47 japhb joined 03:48 greppable6 joined 04:15 japhb joined 04:51 japhb joined 05:17 japhb joined 06:24 bisectable6 joined, evalable6 joined, committable6 joined, bloatable6 joined, quotable6 joined, unicodable6 joined, coverable6 joined, greppable6 joined, benchable6 joined, statisfiable6 joined 06:38 domidumont joined 06:39 leedo_ joined 07:23 domidumont joined 07:28 geekosaur joined 07:29 domidumont joined 07:48 brrt joined 08:00 domidumont joined, robertle joined 08:51 ilmari joined
jnthn morning o/ 08:53
08:55 brrt joined
brrt moarning 08:55
jnthn Ah, smarter yesterday me left Friday me a note on what to do next :) 09:12
samcv hey jnthn 09:21
jnthn o/ samcv 09:24
How'd we end up with that neg codepoint being accessed, ooc?
We construct all strings ourselves...
samcv that is a good question
it only dies when you try and print it to the screen 09:25
well encode the stringc
jnthn What's "it"?
Is there an RT I should be looking at? :)
samcv ok so you decide foā™„o as ascii
and it decodes the ascii but ends up with three synthetics that don't exist
where i assume there are 3 utf-8 bytes
jnthn m: 'foā™„o'.encode('utf-8').decode('ascii') 09:26
camelia Can not decode a utf-8 buffer as if it were ascii
in block <unit> at <tmp> line 1
jnthn m: Buf.new('foā™„o'.encode('utf-8')).decode('ascii')
camelia Will not decode invalid ASCII (code point > 127 found)
in block <unit> at <tmp> line 1
samcv m: with "/tmp/foo2121".IO { .spurt: "foā™„o"; with .open(:enc<ascii>) { say .slurp } }
camelia MoarVM panic: MVM_nfg_get_synthetic_info called with out-of-range synthetic
jnthn That's what should happen
Oh wow
brrt (we're only using the jit_entry_label as a position marker in frame.c and exceptions.c, and that can easily be handled with stack-walking, so we can kill the dynamic labels)
samcv that doesn't show the never error and i changed it to not panic and just throw
with a nicer error message
MVM_nfg_get_synthetic_info call requested a synthetic codepoint that does not exist. 09:27
Requested synthetic 30 when only 6 have been created.
^new message
jnthn Yeah, I think it should oops instead of throw though
And we apparently need to fix our ascii decoder :P
samcv you want it to panic?
jnthn Yes
This should *never* happen, if it did then we've (interanlly) constructed a corrupt string
The user can never give us synthetics directly 09:28
samcv well there are issues with the decoder. better to allow the user to work around a bug that exists
than have it be unrecoverable
i know it should never happen but...
the fact is it does happen :P
maybe you can set it back to panic when the bug isn't triggered anymore? 09:29
not sure how strongly you feel about panicing
though maybe the fix is really simple idk
jnthn I suspect it's a one-line patch 09:31
samcv ah ok. then feel free to set it back to a panic
jnthn Well, can make it oops instead since we get a stack trace that way 09:33
samcv i never got a stack trace on panic
what did i do wrong
jnthn panic doesn't stack trace
oops does
samcv ah ok
jnthn Both exit
panic is for situations where we are in such a bad state we don't know we'll even survive the stack trace :) 09:34
oops is where something really bad has happened, but we expect we can give a stack trace without a SEGV
samcv makes sense 09:35
hmm why did i fail all the nativecall tests 09:36
maybe my build is weird... hmm 09:37
jnthn I've got a heap of other diffs I'm working on in another branch so will do it later, but if you fancy beating me to it, I think the patch is just to change github.com/MoarVM/MoarVM/blob/mast...scii.c#L69 here to declare a MVMuint8 instead of char
samcv idk i have it on --debug=3 --optimize=0 --compiler=clang
i'm going to bed very shortly
jnthn np, I'll do it later today then :)
samcv was gonna finish a last commit
jnthn np 09:38
jnthn is still trying to finish a first commit :)
Geth MoarVM: a3e986924a | (Samantha McVey)++ | 4 files
Fix overflow in uniprop lookups. Closes issue #566

Use MVMint64 instead of MVMGrapheme32 so we don't get an overflow.
In ucd2c.pl: fix some syntax errors. I am not sure why they are showing now and not earlier. This was on Perl v5.24.1		 09:50
MoarVM/spesh-worker: ec38f67c99 | (Jonathan Worthington)++ | src/spesh/candidate.c
Form type tuple from args buffer, not guards.
 09:59
MoarVM/spesh-worker: c53dfcc5aa | (Jonathan Worthington)++ | src/spesh/candidate.c
Remove dead variables; update comments.
 10:02
MoarVM/spesh-worker: 37167b3991 | (Jonathan Worthington)++ | src/spesh/dump.c
Remove old arg guards from spesh dumper.
 10:06
MoarVM: 4ff2f1f918 | (Samantha McVey)++ | src/strings/normalize.c
Improvements to segmentation of Emoji w/ GCB=Other

Not all Emoji Modifiers have Grapheme_Cluster_Break = E_Base or E_Base_GAZ. In these cases we need to check the Emoji_Modifier_Base property.
Correctly segments 25 more Emoji than before (Unicode Emoji v4.0).		 10:09
samcv ok. i'm done for tonight :) time to go to bed
night all o/
Zoffix night 10:10
jnthn 'night, samcv++ 10:12
samcv ok added a roast test. now i can really sleep 10:20
jnthn :) 10:22
Geth MoarVM/spesh-worker: f62474bda8 | (Jonathan Worthington)++ | 5 files
Eliminate old arg guards mechanism fully.
 10:24
11:05 brrt joined 11:11 brrt joined
Geth MoarVM/spesh-worker: ae86b1c710 | (Jonathan Worthington)++ | 3 files
Make arg specialization take a type tuple.

Rather than the args buffer. This will make it easier to migrate to use from the specialization worker thread, and also to produce specializations that are not type-sensitive or only commit to certain types.		 11:16
jnthn lunch & 11:17
11:25 brrt joined
jnthn back 12:13
OK, so...new logging is in, new stats model is in, new guards are in 12:28
So the next step is the planner 12:30
(The thingy that decides what specializations to build.)
timotimo say, for baseline spesh we don't really need anything more than just the planner doing the spesh earlier and not having anything happen for args? 12:31
oh, not quite true
it does callsite-based opts of course
jnthn Also wvals and static lexicals provide info that we can rely on 12:33
timotimo ah, indeed 12:34
jnthn Though today I use the term "certain" rather than "baseline"
timotimo sure
jnthn Note that if we see that every call is, say, (Int,Int) or so, there's no reason to produce a certain one
timotimo fair enough, yeah 12:35
Geth MoarVM/spesh-worker: dc80d7def6 | (Jonathan Worthington)++ | 5 files
Sketch out specialization plan data structure.
 13:05
13:31 AlexDaniel joined
Geth MoarVM/spesh-worker: 3e6890c11e | (Jonathan Worthington)++ | 4 files
Add spesh plan memory management bits.
 13:35
[Coke] spesh question; you make faster code when you notice types don't change, partly by removing type checks; how do you then fall back when the types DO change? 13:37
timotimo we emit guard operations every time we assume something
when the guard fails we do a "deopt"
[Coke] is the guard.. a ... type check? 13:38
I'm trying to suss how it can be cheaper than the original work.
ilmari [Coke]: the code following the guard can be specialised for that type without further checking 13:39
instead of each op having to check/dispatch on the type
[Coke] ilmari: ah, thanks. 13:42
jnthn Right, it's a "check it once, then rip out all further checks" kind of deal
Except we tend to do precise checks too 13:43
Rather than "is it some kind of..."
Which are also cheaper (just a pointer comparison rather than an MRO search)
It's not just type checks, though; from that we can also do stuff like ripping out pointless decont ops, or turning decont ops we need into cheaper operations without the virtual calls 13:44
[Coke] jnthn: with the utf8/strict discussion I saw earlier, does that positively influence your million-line-read bench?
jnthn [Coke]: Didn't do any new measuring, but it's probable you can get Perl 5 to go faster if you know what you're doing.
Rather than taking the typical approach of :encoding(utf-8) or so 13:45
So it probably speeds up the Perl 5 case and gives us some further catching up :P
ilmari :utf8 would be faster, but unsafe
because it doesn't actually decode or check, just flags it as utf8 internally 13:46
jnthn omg
timotimo hah
that's just mean
ilmari yeah, it's heavily discouraged
jnthn But wait, that means that everything you do downstream must have to check "is this OK utf8"?
Or is that amortized?
[Coke] I just wanted to make sure we weren't missing a chance to force a stricter, slower mode to compare apples. 13:47
ilmari it means everything breaks if it's not actually valid
timotimo i don't think anything checks if "is this valid utf8"
jnthn Breaks as in...exception? SIGSEGV?
[Coke] er, *.apples?
ilmari on a debug build you might get assert failures
jnthn: sigsegvs, bogus results, panics, all bets are off 13:48
timotimo [Coke]: many operations are implemented as a lookup on the REPR of the object and the REPR has a bunch of so called REPRops, which are basically function pointers
13:48 brrt joined
jnthn ilmari: Goodness. o.O 13:48
No wonder it's discouraged!
timotimo [Coke]: but if we know the exact type, we also know the REPR and all of its contents, so instead of multiple pointer dereferences and a function call invocation we can be much smarter
ilmari I believe leont and others are working on a :utf8-strict layer, which actuall checks
jnthn Yeah, that's probably the one to compare with then
ilmari but is faster than :encoding(UTF-8)
[Coke] glad you smart people are working on these bits. :)
ilmari for now, :encoding(UTF-8) is the right thing to compare to
jnthn :utf-8 is apples and oranges
timotimo and some REPRs will do some extra optimization on top of that. for example, Rakudo Scalar can grab its contents with a "dereference this pointer and grab the pointer that lives at this offset" operation, which is extremely cheap (especially when jitted) 13:49
jnthn Uh, :utf8 rather
timotimo jnthn: have you been comparing to :utf8 or :encoding(UTF-8)?
jnthn timotimo: The latter :)
timotimo OK, so no cheap win to be had there :) 13:50
[Coke] *finger snap*
timotimo indeed
jnthn So basically, utf8-strict sounds like it will give us some more serious competition.
But if :encoding(UTF-8) is typical today then the 1.1x result stands as representative of real world use. 13:51
And we'll find that .1 :)
timotimo i hear thunder. i hope it'll really come down soon and cool things off 13:56
jnthn We're due some. I'd welcome it. 13:57
It's managed to get a few degress warmer than was originally forecast today 14:00
timotimo it says something like 32degC max
i'm having trouble cooling down my apartment
moritz we had a thunderstorm at like 4am this morning :-) 14:01
jnthn It's 30 here right now
timotimo last night i had one window open and put a fan in the way, but then a gigantic insect came in
it was about as long as my index finger i'm sure
looked like a grasshopper, except i think those are more like fingernail sized? so maybe it was a locust?!
jnthn Hah, we had one of those come in last summer
Long green thing. My wife thought it was a locust.
timotimo yeah, sounds like that's what i had 14:02
jnthn It slept on the curtain rail overnight. I nicknamed it Mr Greeny. Flew away out the window the next morning.
timotimo i do have a picture of it, but i had the lights off and the flash decided to light only the curtain, not the insect, so it's really dark
let me see if i can "enhance"
looks better 14:03
imgur.com/1fl6sEt
jnthn Hm, looks pretty similar 14:04
'twas big, but given it just hung out overnight and left, I'd prefer it to mosquitos, which bite me :P 14:05
timotimo i'm unreasonably squeamish when it comes to insects
jnthn Yeah, can understand it
timotimo i was very glad when it decided to fly out the window and sit against the blinds, then i closed the window on 'em
jnthn I'm not that keen, tbh, but manage not to be too bothered
timotimo i'm scared i might have trapped it there and when i open the window it'll come in and be angry at me 14:06
uh oh, i crushed it ... i think it's dead 14:10
jnthn oops
Geth MoarVM/spesh-worker: 5952915f3e | (Jonathan Worthington)++ | 5 files
Stub in specialization planning and dumping.

No calculations yet; this is just the groundwork.		 14:21
MoarVM/spesh-worker: 371220a983 | (Jonathan Worthington)++ | 3 files
Aggregate OSR hits over all callsites.
 14:29
14:35 brrt joined
timotimo at the moment i get - for a quite long test file - just a single plan and it says "0 specializations will be produced" 14:45
jnthn Right, as the commit said, no calculations done yet ;) 14:52
timotimo oh
duh :)
jnthn Also we only ever send one buffer of stats so far
14:56 lizmat joined
Geth MoarVM/spesh-worker: 0cfd3bb8f0 | (Jonathan Worthington)++ | 2 files
Implement basic planning by callsite.

This just adds in certain specializations; types are not considered at this point.		 15:04
15:04 domidumont joined
MoarVM/spesh-worker: 08c31e644b | (Jonathan Worthington)++ | src/spesh/dump.c
Explain why certain specializations were produced.
timotimo the rain has arrived 15:05
aaaaand it's gone 15:06
jnthn Not here yet :(
The type analysis comes next, but it's a bit warm and late on a Friday afternoon to figure that out. 15:07
I'll take a look at that ASCII decoder fail from earlier though
Am taking a couple of days vacation at the start of next week, also, so will continue with the spesh stuff on Wed when I'm back 15:08
(Goes without saying this stuff isn't aimed for the 2017.07 release)
timotimo that sounds nice
dogbert17 the ASCII deocder fail was reported by Zoffix as RT #131384 15:14
synopsebot6 Link: rt.perl.org/rt3/Public/Bug/Display...?id=131384
dogbert17 wonders what a deocder is 15:16
jnthn Something that looks at a bunch of bytes and works out what unicode codepoints they represent 15:18
Oh, hah, the typo :P
dogbert17 :)
Geth MoarVM: 31ca7fa702 | (Jonathan Worthington)++ | src/strings/ascii.c
Fix ASCII streaming decode error handling.

We accidentally used a signed type, and so would treat out of range as if they were synthetics, rather than simply giving the out of range error.		 15:35
MoarVM: 4d408e9abf | (Jonathan Worthington)++ | src/strings/nfg.c
Make illegal synthetic an oops, not a throw.

This partially undoes f621f21191c5c3a3035d3121c042cba4348c18cc, but leaves in place the stack trace. The reasoning is that an invalid synthetic is almost certainly going to be an indication of a serious internal error (potentially corruption); only MoarVM constructs synthetic codepoints, so any time an invalid one shows up it has to be a VM mistake.
MoarVM: 45b008fb6f | (Jonathan Worthington)++ | src/core/exceptions.c
Tweak a comment and oops output.
16:04 robertle joined 16:09 travis-ci joined
travis-ci MoarVM build errored. Jonathan Worthington 'Tweak a comment and oops output.' 16:09
travis-ci.org/MoarVM/MoarVM/builds/251204892 github.com/MoarVM/MoarVM/compare/4...b008fb6ff8
16:09 travis-ci left 16:15 bisectable6 joined 16:50 domidumont joined 17:10 bisectable6 joined 17:18 buggable joined 17:21 bisectable6 joined 17:46 evalable6 joined 19:55 FROGGS joined
Geth MoarVM: 522e005ced | (Samantha McVey)++ | src/strings/normalize.c
Actually fix the emoji from commit 4ff2f1f9

There was an error in the location of the previous commit, this moves it to the proper place.		 21:02
samcv oops looks like i pushed the wrong thing eariel
well there it's fixed 21:03
21:15 MasterDuke joined
MasterDuke timotimo, jnthn, et al: might find this useful, a gui for perf results similar to heaptrack: www.kdab.com/hotspot-gui-linux-perf-profiler/ 21:19
timotimo doesn't look half bad 21:20
MasterDuke just installed it on my desktop, haven't had a chance to try it out yet, but it does look promising 21:22
21:24 evalable6 joined 21:29 travis-ci joined
travis-ci MoarVM build passed. Samantha McVey 'Actually fix the emoji from commit 4ff2f1f9 21:29
travis-ci.org/MoarVM/MoarVM/builds/251309821 github.com/MoarVM/MoarVM/compare/4...2e005ced36
21:29 travis-ci left
jnthn MasterDuke: Looks nice; will give it a try :) 22:34
Geth MoarVM: 06819cce28 | (Samantha McVey)++ | src/strings/normalize.c
Use GCB instead of HST to avoid property lookup for Hangul

This should result in some speed up and we don't have to use strcmp anymore and we can just compare integers. It doesn't require any more property lookups than any other codepoints going through should_break() now.		 22:35
jnthn samcv++ # nice! Bet that generates far nicer code too :) 22:38
samcv yep 22:40
jnthn Do you think the maybe_hangul range checks are still a win? 22:43
Hm, they maybe are, looking at it 22:44
Guess it'd need callgrind to tell us :)
samcv hmm well. probably not needed maybe. hm 22:48
heh
22:49 colomon_ joined
samcv this code though github.com/MoarVM/MoarVM/blob/0681...#L338-L339 i-im not sure about the strlen(ccc_str) > 3 part 22:52
so it returns 0 if there's no string, otherwise returns 0 if the strings length is greater than 3 22:53
because?
about the returning 0 if it's greater than 3
timotimo so the segfault inside MVMCallCapture's gc_mark, right?
samcv the one in my Font::QueryInfo module?
that is where it crashes
timotimo i've got it open in rr
from the whateverables
i see its data being overwritten in the utf8-c8 encoder, but that's just because it had been freed and alloced in between 22:54
we have this "owns_callsite" flag that tells us whether or not it's okay to free the thing 22:55
and if it's set, we'll memcpy and everything when we pass the thing around, right?
jnthn samcv: There are (were?) some values of the CCC property that are not integers
I forget exactly what it is but it's something like Unassigned or some such
The > 3 check is for that :)
samcv hehehe 22:56
timotimo m: say Buf.new(:16<69 2f 32 64 31 33 35 63>).decode("utf8")
camelia ===SORRY!=== Error while compiling <tmp>
Malformed radix number
at <tmp>:1
------> say Buf.new(:16<69ā 2f 32 64 31 33 35 63>).decode("utf8")
expecting any of:
number in radix notation
timotimo m: say Buf.new(0x69 ,0x2f, 0x32, 0x64, 0x31, 0x33, 0x35, 0x63>).decode("utf8") 22:57
camelia ===SORRY!=== Error while compiling <tmp>
Missing required term after infix
at <tmp>:1
------> x2f, 0x32, 0x64, 0x31, 0x33, 0x35, 0x63>ā).decode("utf8")
expecting any of:
prefix
term
timotimo m: say Buf.new(0x69 ,0x2f, 0x32, 0x64, 0x31, 0x33, 0x35, 0x63).decode("utf8")
camelia i/2d135c
timotimo hm, that's probably not a valid string?
samcv jnthn, looks like Not_Reordered 22:58
jnthn samcv: I guess the presence of those is why it's a str property rather than an int property anyway
Ah, right, that's the one :)
It's been more than 2 years :)
timotimo so how do i go about debugging this? i can set watchpoints to find where it gets calloced (for the string to be put in and also for the callsite to be put in), i can get a breakpoint where the string content is actually written, but how does that help me? :|
hm. maybe i can watchpoint the pointer in the arg proc context to see when the callsite gets assigned 23:04
maybe callsites should become GC-managed objects? :\
samcv looks like Not_Reordered jnthn 23:08
oh i said that 23:09
heh
was gonna change this but it seems to break things 23:12
to int
err wait no maybe that was another section. going to only make this one change and run a full spectest 23:16
dogbert17 timotimo: night time bug hunting ?
timotimo i think i need to go to bed and maybe try again tomorrow 23:17
dogbert17 yeah, is about time for me as well, doing an experiment with samcv's font program 23:18
dogbert17 wonders how long time this program is supposed to take 23:19
samcv oh yay i got it to work jnthn :) with int based ccc 23:21
\o/
dogbert17 en => NanumBarunGothic ? 23:22
samcv: your font program seems to write out a gazillion backtraces. Is it supposed to do that? 23:27
and there it completed 23:28
samcv backtraces?
how so? 23:29
dogbert17 malformed base-16 number
23:29 evalable6 joined, bloatable6 joined, quotable6 joined, committable6 joined, benchable6 joined, coverable6 joined, bisectable6 joined, unicodable6 joined, greppable6 joined
samcv that shouldn't happen 23:29
23:29 statisfiable6 joined
samcv what system are you on 23:29
dogbert17 32 bit Linux Mint
samcv dogbert17, can you run 'prove -e perl6'
23:30 committable6 joined
dogbert17 sec 23:30
test failures 23:31
gist.github.com/dogbert17/cc7bd2da...e593d658f0 23:32
it looked like that when running your program as well
timotimo source => "!!#0H|>^1!|>^1!P0oWQ" - ?!?!? 23:34
dogbert17 :)
timotimo anyway, bedtime.
o/
dogbert17 I might possibly have a clue as to what's going on
samcv: does it always crash for you? 23:35
samcv every time
dogbert17 try this
samcv ah yes
there's some problem with some freetypes
travis did that too
dogbert17 MVM_SPESH_DISABLE=1 perl6 ... 23:36
samcv dogbert17, edit the sample program
to not get charset
one sec
font-query-all($file, 'charset')
that should do everything but charset 23:37
and not show those errors
Geth MoarVM: 90e5157766 | (Samantha McVey)++ | src/strings/normalize.c
Reorder conditional in maybe_hangul() to be more clear

This makes it clear at a glance the ranges of the conditionals which cause it to return true.		 23:40
MoarVM: e92ea5a35a | (Samantha McVey)++ | src/strings/normalize.c
Speed up ccc by making ccc_relative using integer prop lookup

Gets the canonical combining class for a codepoint. Does a shortcut since CCC is stored as a string property, though because they are all sorted numerically it is ok to get the internal integer value as stored instead of the string.
Will result in a speedup during normalization.
dogbert17 I changed to 'say $_ => font-query-all($_, 'charset') ...' was that what you meant? 23:41
samcv dogbert17, i just pushed changes 23:44
pull the latest changes and then yeah
set it like that
i just fixed a bug in making exceptions
dogbert17 now the exceptions are gone :) 23:46
samcv yay
what version do you have btw?
fc-query --version
dogbert17 fontconfig version 2.11.0 23:47
looks good so far 23:50
samcv wow hmm it's not crashing 23:51
it crashes if i have it query the charset though
dogbert17 not if you run it with MVM_SPESH_DISABLE=1 23:52
samcv did you get a segfault when running WITH the charset?
does it throw errors or does it just kill the program?
dogbert17 no, only lots of what you saw in the gist
samcv and no crashing?
yeah but the program didn't crash when you ran Query-all-fonts.p6 or whatever? 23:53
like. due to the exception i mean
dogbert17 it crashes if I don't disable SPESH
samcv ok
dogbert17 give it a try
samcv no crashing 23:54
dogbert17 :)
23:59 travis-ci joined
travis-ci MoarVM build failed. Samantha McVey 'Speed up ccc by making ccc_relative using integer prop lookup 23:59
travis-ci.org/MoarVM/MoarVM/builds/251352717 github.com/MoarVM/MoarVM/compare/0...2ea5a35abb
23:59 travis-ci left

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!