[08:52] *** harrow left
[09:10] *** harrow joined
[11:35] <ShimmerFairy> Hmm... when it comes to Unicode collation, am I right in thinking that our ring buffer implementation is designed to avoid processing the whole string, by keeping a narrow moving window into both strings instead? (I'm only just starting to dig into the collation stuff, because Unicode 18 has made some changes to the UCA.)

[11:37] <lizmat> samcv would know, as she did most of that work

[11:38] <lizmat> but that assumption feels right

[11:40] <ShimmerFairy> I ask because there's a new rule that essentially allows U+FFFE to act as a way of separating different parts of a string for the purposes of collation (they give an example of "Last Name\x[FFFE]First Name", to give an idea). And given what little I've read of UTS#10 so far, I get the bad feeling our UCA impl needs to be overhauled, if not rewritten.

[11:43] <lizmat> that would be...  a major undertaking  :-(

[11:44] <lizmat> or could It be handled by first splitting on \xFFFE ?

[11:44] <ShimmerFairy> m: say "l\x[FFFE]f" unicmp "l f" # this should be Less, since 'l' is a prefix of 'l f'

[11:44] <camelia> rakudo-moar e17a1d43b: OUTPUT: «More␤»

[11:45] <lizmat> well, FFFE is a non-character, so I guess then it will just take the codepoint value and that would always be more ?

[11:48] <ShimmerFairy> I'm not sure splitting works, but I need to read up on the UCA to be able to make many comments. I've only seen enough to get that bad feeling. I believe the "Merging Sort Keys" section of UTS#10 illustrates why splitting the string on U+FFFE won't work.

[11:48] <ShimmerFairy> https://www.unicode.org/reports/tr10/tr10-55.html#Merging_Sort_Keys

[11:50] <lizmat> yuck

[11:53] <ShimmerFairy> I think if we'd rather get Unicode 18.0 support out the door ASAP, then I can have the spectest mark off the FFFE tests as "todo" and just leave it broken (since this is a new feature to the UCA, nobody is using it right now to collate those kinds of "multipart" strings).

[11:53] <ShimmerFairy> There's also changes to the implicit weights, but that seems like a much simpler fix (and also seems to have been outdated for a while, oops).

[11:56] <lizmat> that feels like a plan, fwiw

[11:56] <lizmat> but maybe not for the 2026.09 release ?

[12:00] <ShimmerFairy> Yeah, as much as I'd like to believe in myself, I don't think I could get Unicode 18.0 support in in time for September if I insisted on fixing the U+FFFE bit in the UCA as a prereq. I have to understand the UCA *and* samcv's implementation *and* possibly rip up large parts of said impl.

[12:00] <lizmat> understood and ++ShimmerFairy 

[12:00] <ShimmerFairy> Having the brand new collation feature on U+FFFE be broken for a month seems like a reasonable tradeoff for having all the rest of Unicode 18 available.

[12:01] * lizmat agrees

[12:16] *** MasterDuke joined
[12:17] <MasterDuke> ShimmerFairy, japhb: if you're working on unicode stuff, https://github.com/MoarVM/MoarVM/commit/0cbe0cd8996688b4a22e207721e63a91eed07812#commitcomment-175745918 and https://github.com/MoarVM/MoarVM/commit/18c880107e5ce3be2f670426d72a04be9eb24d6f#commitcomment-175746820 might be relevant

[12:21] <ShimmerFairy> I'll keep that in mind. There's definitely funky stuff going on, and a number of our tests made the decision to allow for a certain amount of error to creep in (look at some of the test files in rakudo's own personal test suite), which likely hasn't helped.

[12:23] *** MasterDuke left
[16:20] *** Nicholas left
[16:41] <[Coke]> we don't need to rush it

[18:19] *** Nicholas joined
[23:10] <Geth> ¦ MoarVM: ugexe++ created pull request #2044: Let spesh see the object at SC index 0, speeding up RakuAST

[23:10] <Geth> ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/2044