|
08:52
harrow left
09:10
harrow joined
|
|||
| ShimmerFairy | Hmm... when it comes to Unicode collation, am I right in thinking that our ring buffer implementation is designed to avoid processing the whole string, by keeping a narrow moving window into both strings instead? (I'm only just starting to dig into the collation stuff, because Unicode 18 has made some changes to the UCA.) | 11:35 | |
| lizmat | samcv would know, as she did most of that work | 11:37 | |
| but that assumption feels right | 11:38 | ||
| ShimmerFairy | I ask because there's a new rule that essentially allows U+FFFE to act as a way of separating different parts of a string for the purposes of collation (they give an example of "Last Name\x[FFFE]First Name", to give an idea). And given what little I've read of UTS#10 so far, I get the bad feeling our UCA impl needs to be overhauled, if not rewritten. | 11:40 | |
| lizmat | that would be... a major undertaking :-( | 11:43 | |
| or could It be handled by first splitting on \xFFFE ? | 11:44 | ||
| ShimmerFairy | m: say "l\x[FFFE]f" unicmp "l f" # this should be Less, since 'l' is a prefix of 'l f' | ||
| camelia | More | ||
| lizmat | well, FFFE is a non-character, so I guess then it will just take the codepoint value and that would always be more ? | 11:45 | |
| ShimmerFairy | I'm not sure splitting works, but I need to read up on the UCA to be able to make many comments. I've only seen enough to get that bad feeling. I believe the "Merging Sort Keys" section of UTS#10 illustrates why splitting the string on U+FFFE won't work. | 11:48 | |
| www.unicode.org/reports/tr10/tr10-..._Sort_Keys | |||
| lizmat | yuck | 11:50 | |
| ShimmerFairy | I think if we'd rather get Unicode 18.0 support out the door ASAP, then I can have the spectest mark off the FFFE tests as "todo" and just leave it broken (since this is a new feature to the UCA, nobody is using it right now to collate those kinds of "multipart" strings). | 11:53 | |
| There's also changes to the implicit weights, but that seems like a much simpler fix (and also seems to have been outdated for a while, oops). | |||
| lizmat | that feels like a plan, fwiw | 11:56 | |
| but maybe not for the 2026.09 release ? | |||
| ShimmerFairy | Yeah, as much as I'd like to believe in myself, I don't think I could get Unicode 18.0 support in in time for September if I insisted on fixing the U+FFFE bit in the UCA as a prereq. I have to understand the UCA *and* samcv's implementation *and* possibly rip up large parts of said impl. | 12:00 | |
| lizmat | understood and ++ShimmerFairy | ||
| ShimmerFairy | Having the brand new collation feature on U+FFFE be broken for a month seems like a reasonable tradeoff for having all the rest of Unicode 18 available. | ||
| lizmat agrees | 12:01 | ||
|
12:16
MasterDuke joined
|
|||
| MasterDuke | ShimmerFairy, japhb: if you're working on unicode stuff, github.com/MoarVM/MoarVM/commit/0c...-175745918 and github.com/MoarVM/MoarVM/commit/18...-175746820 might be relevant | 12:17 | |
| ShimmerFairy | I'll keep that in mind. There's definitely funky stuff going on, and a number of our tests made the decision to allow for a certain amount of error to creep in (look at some of the test files in rakudo's own personal test suite), which likely hasn't helped. | 12:21 | |
|
12:23
MasterDuke left
16:20
Nicholas left
|
|||
| [Coke] | we don't need to rush it | 16:41 | |
|
18:19
Nicholas joined
|
|||
| Geth | MoarVM: ugexe++ created pull request #2044: Let spesh see the object at SC index 0, speeding up RakuAST |
23:10 | |