Geth rakudo: 264a1a27eb | (Zoffix Znet)++ | 2 files
Fix `where` optimization for Junction args

Phixes github.com/rakudo/rakudo/issues/1457
Our optimization uses `istype` op, which doesn't thread, so with unthreaded params, instead of doing a smartmatch with a Junction, we end up doing istype on it and blowing up.
Fix by annotating unthreaded params and using a slightly more elaborate optimization for them, where we check if the arg is a Junction and use the slower-path ACCEPTS call in those cases.		 00:03
rakudo: 264a1a27eb | (Zoffix Znet)++ | 2 files
Fix `where` optimization for Junction args

Phixes github.com/rakudo/rakudo/issues/1457
Our optimization uses `istype` op, which doesn't thread, so with unthreaded params, instead of doing a smartmatch with a Junction, we end up doing istype on it and blowing up.
Fix by annotating unthreaded params and using a slightly more elaborate optimization for them, where we check if the arg is a Junction and use the slower-path ACCEPTS call in those cases.		 00:04
TimToady seems like a dup notificaation, mebbe we have two 00:06
geths running? 00:07
Geth rakudo: 264a1a27eb | (Zoffix Znet)++ | 2 files
Fix `where` optimization for Junction args

Phixes github.com/rakudo/rakudo/issues/1457
Our optimization uses `istype` op, which doesn't thread, so with unthreaded params, instead of doing a smartmatch with a Junction, we end up doing istype on it and blowing up.
Fix by annotating unthreaded params and using a slightly more elaborate optimization for them, where we check if the arg is a Junction and use the slower-path ACCEPTS call in those cases.
TimToady er...
Geth rakudo: 264a1a27eb | (Zoffix Znet)++ | 2 files
Fix `where` optimization for Junction args

Phixes github.com/rakudo/rakudo/issues/1457
Our optimization uses `istype` op, which doesn't thread, so with unthreaded params, instead of doing a smartmatch with a Junction, we end up doing istype on it and blowing up.
Fix by annotating unthreaded params and using a slightly more elaborate optimization for them, where we check if the arg is a Junction and use the slower-path ACCEPTS call in those cases.		 00:08
TimToady if 3 times is enemy action, what's 4 times? 00:09
Zoffix .
Was geth printing stuff?
TimToady 4 times
same report
Zoffix oops sorry. I was looking at the log and nothing was there. I then looked at web hook and it tells it "failed to deliver" due to timeout
Geth roast: 8920e85840 | (Zoffix Znet)++ | S02-types/subset.t
Cover `where` clauses with Junction args

Closes github.com/rakudo/rakudo/issues/1457 Rakudo fix: github.com/rakudo/rakudo/commit/264a1a27eb		 00:10
TimToady maybe we should make geth do the multi-colored confettit thing like other spammers so we can ignore it easily :) 00:11
Zoffix :D
TimToady *fetti 00:12
Geth rakudo: 6266fd53b9 | (Zoffix Znet)++ (committed using GitHub Web editor) | src/Perl6/Optimizer.nqp
Fix typo in comment; MasterDuke++
 00:37
Zoffix m: use MONKEY; augment class Mu { method test { dd nqp::iscont(self) } }; (Proxy.new: :STORE{$^a, $^b}, :FETCH{Int;}).VAR.test 00:47
camelia 1
Zoffix so there *are* cases when `self` is conted..
m: use MONKEY; augment class Mu { method meow(Mu:D) { }; method test(Mu:D) { self.meow } }; (Proxy.new: :STORE{$^a, $^b}, :FETCH{Int;}).VAR.test 00:54
camelia Too few positionals passed; expected 2 arguments but got 1
in method test at <tmp> line 1
in block <unit> at <tmp> line 1
Zoffix m: use MONKEY; augment class Mu { method meow(Mu:D:) { }; method test(Mu:D:) { self.meow } }; (Proxy.new: :STORE{$^a, $^b}, :FETCH{Int;}).VAR.test 00:55
camelia No such method 'meow' for invocant of type 'Int'. Did you mean any of these?
meow
new

in method test at <tmp> line 1
in block <unit> at <tmp> line 1
Zoffix hehehe :) It's a :D and not :D :)
ZofBot: quantum :D
ZofBot Zoffix, She was sitting bolt upright in a chair, staring into space
Geth rakudo: 902f45f508 | (Zoffix Znet)++ | src/core/Mu.pm
Fix crash in Proxy.perl

Phixes github.com/rakudo/rakudo/issues/1466
The crash occurs due to `self` of a Proxy-ed object being conted, but access to it fetches the value. So we end up entering .perl via Mu:D candidate (the Proxy object itself) and then calling ... (8 more lines)		 01:17
roast: 625e80d31b | (Zoffix Znet)++ | S06-routine-modifiers/proxy.t
Remove trailing whitespace
roast: 6796c19f61 | (Zoffix Znet)++ | S06-routine-modifiers/proxy.t
Cover crashes in Proxy.perl

Closes github.com/rakudo/rakudo/issues/1466 Rakudo fix: github.com/rakudo/rakudo/commit/902f45f508
Unsure whether it should give a "Proxy.new: …" value rather than the .perl of the fetched value, so speccing only that the construct doesn't explode.		 01:19
nqp: 34671bd59f | (Samantha McVey)++ | tools/build/MOAR_REVISION
Bump MoarVM to get support for windows-1251 (Cyrillic)

Changes: 2018.01-53-g7fdc6aa..2018.01-55-g823cd43dd 823cd43dd Add windows-1251 (Cyrillic) decode/encode. Fix bug in windows-1252 6306b824a Add script to generate Windows-1251 and 1252 codetables		 01:46
¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...g823cd43dd
rakudo: c73cb78fba | (Samantha McVey)++ | 2 files
Add windows-1251 encoding to the list of possible encodings

Also allow windows1251 and windows1252 (without the dash) to work as well.		 01:47
rakudo: 4507a56541 | (Samantha McVey)++ | tools/build/NQP_REVISION
Bump Moar/NQP to get support for windows-1251 (Cyrllic) encoding

MoarVM Changes: 2018.01-53-g7fdc6aa..2018.01-55-g823cd43dd 823cd43dd Add windows-1251 (Cyrillic) decode/encode. Fix bug in windows-1252 6306b824a Add script to generate Windows-1251 and 1252 codetables
rakudo: version bump brought these changes: github.com/perl6/nqp/compare/2018....g34671bd59
jstuder-gh++ created pull request #1467: Issue 1341: Elevate statevars to enclosing block when wrapped
samcv yay. though there is a bug in windows-1251 and 1252 where it will pass through codepoints which don't exist in that codemap 01:50
but that was a bug before
m: Buf.new(0x81).decode('windows-1252') # 0x81 doesn't exist for example in this table 01:52
camelia ( no output )
samcv and perl 5 and other decoders will reject it
tbrowder AlexDaniel: what about NQP PR #401? Any problems with it? 02:30
roast: jstuder-gh++ created pull request #383:
Rakudo Issue 1341: Statevar not reset after do statement prefix		 04:35
nqp/sha1bin: 46a8d3c4cb | (Stefan Seifert)++ | src/vm/moar/QAST/QASTOperationsMAST.nqp
Map new sha1bin op
 06:04
rakudo/sha1bin: 67a70cff44 | (Stefan Seifert)++ | 5 files
Use new sha1bin op for added performance
 06:06
Kaiepi n 06:17
wrong window sorry 06:18
nine Why....do we use the current time at object creation as $!desc of a VM object?? 08:41
[Tux] Rakudo version 2018.01-77-g4507a5654 - MoarVM version 2018.01-58-g8a3bcd9a4
csv-ip5xs0.983 - 0.996
csv-ip5xs-2010.658 - 10.732
csv-parser12.208 - 12.294
csv-test-xs-200.442 - 0.451
test9.540 - 9.572
test-t2.585 - 2.587
test-t --race1.105 - 1.152
test-t-2047.088 - 49.662
test-t-20 --race16.426 - 17.351
 09:32
Geth rakudo: 299e85264b | (Jeremy Studer)++ | src/Perl6/Actions.nqp
Elevate statevars to enclosing block when wrapped

Previously in blocks following the "do" statement prefix, statevars would be seemingly reset after each iteration of the loop.
Given the statement:
... (32 more lines)		 09:37
rakudo: 646266ae70 | (Zoffix Znet)++ (committed using GitHub Web editor) | src/Perl6/Actions.nqp
Merge pull request #1467 from jstuder-gh/state_deep_cloning

Issue 1341: Elevate statevars to enclosing block when wrapped
roast: 6788136aa4 | (Jeremy Studer)++ | S04-declarations/state.t
Statevar not reset after do statement prefix

Test that statevar values in blocks following "do" statement prefix is not reset on each iteration.
  See [Rakudo Issue 1341](github.com/rakudo/rakudo/issues/1341)
roast: 1f7d29ff4f | (Zoffix Znet)++ (committed using GitHub Web editor) | S04-declarations/state.t
Merge pull request #383 from jstuder-gh/state_deep_cloning

Rakudo Issue 1341: Statevar not reset after do statement prefix
rakudo: 86e926c76d | (Stefan Seifert)++ | src/core/VM.pm
Use a more descriptive desc for $*VM

Using the current date and time is hardly descriptive and costs ~ 10ms when loading a native library (which needs $*VM).		 10:08
travis-ci Rakudo build failed. Zoffix Znet 'Merge pull request #1467 from jstuder-gh/state_deep_cloning 10:15
travis-ci.org/rakudo/rakudo/builds/336476414 github.com/rakudo/rakudo/compare/4...6266ae70a6
buggable [travis build above] ☠ Did not recognize some failures. Check results manually.
Zoffix weird. master of all threee repos build fine on my VM 10:28
Zoffix tries `z bump` 10:29
.ask samcv did you by chance miss pushing some commits? Travis says current Rakudo build fails to compile, yet master of all repos compiles just fine for me. I tried doing bumps, but there's a bunch of encoding-related failures in t/spec/S16-filehandles/io.t and a crash in t/spec/S32-str/encode.t with "Error encoding Windows-1252 string: could not encode codepoint 129" 10:37
yoleaux Zoffix: I'll pass your message to samcv.
Zoffix I guess I'll push the bumps, to see if that at least makes travis compile 10:38
Geth nqp: 99493b06b3 | (Zoffix Znet)++ | tools/build/MOAR_REVISION
[MoarVM Bump] Brings 3 commits

MoarVM bump brought: github.com/MoarVM/MoarVM/compare/2...8-g8a3bcd9 8a3bcd9 Throw on invalid windows-1251/1252 during encode and decode b389521 Refactor so code is not duped between windows-1251 & 1252 3bae45b Merge pull request #793 from samcv/windows-1251
¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...3cd43dd... No newline at end of file
rakudo: e1a8b1d907 | (Zoffix Znet)++ | tools/build/NQP_REVISION
[NQP Bump] 99493b0 [MoarVM Bump] Brings 3 commits

NQP bump brought: github.com/perl6/nqp/compare/2018....3-g99493b0
MoarVM bump brought: github.com/MoarVM/MoarVM/compare/2...8-g8a3bcd9 8a3bcd9 Throw on invalid windows-1251/1252 during encode and decode b389521 Refactor so code is not duped between windows-1251 & 1252 3bae45b Merge pull request #793 from samcv/windows-1251
rakudo: version bump brought these changes: github.com/perl6/nqp/compare/2018....671bd59... No newline at end of file
Kaiepi++ created pull request #1468: Add more documentation on building MoarVM
Zoffix nope, still fails to compile on travis :S 10:49
Weird. I guess other suspect would be github.com/rakudo/rakudo/commit/64...778840fb1d
Zoffix waits for samcv++'s reply before trying to revert that one
travis-ci Rakudo build failed. Stefan Seifert 'Use a more descriptive desc for $*VM
travis-ci.org/rakudo/rakudo/builds/336486153 github.com/rakudo/rakudo/compare/6...e926c76d6b
buggable [travis build above] ☠ Did not recognize some failures. Check results manually. 10:50
rakudo: 66f1690f52 | (Ben Davies)++ | 2 files
Add more documentation on building MoarVM
 10:55
rakudo: ddda743974 | (Zoffix Znet)++ (committed using GitHub Web editor) | 2 files
Merge pull request #1468 from Kaiepi/docs

Add more documentation on building MoarVM
travis-ci Rakudo build failed. Zoffix Znet '[NQP Bump] 99493b0 [MoarVM Bump] Brings 3 commits 11:28
travis-ci.org/rakudo/rakudo/builds/336496077 github.com/rakudo/rakudo/compare/8...a8b1d90758
buggable [travis build above] ☠ Did not recognize some failures. Check results manually.
Geth rakudo: d80fc376d2 | (Zoffix Znet)++ | src/Perl6/Actions.nqp
Fix whatever curry with regexes

Fixes RT#131409: rt.perl.org/Ticket/Display.html?id=131409
When we curry the QAST, we migrate blocks in chains, but we forget to migrate blocks created by constructs inside Regexes.
Fix by adding a migrator to the branch that handles these ops.		 11:36
synopsebot RT#131409 [open]: rt.perl.org/Ticket/Display.html?id=131409 whatever auto closure fails with smartmatch with regex
roast: 323a2b6594 | (Zoffix Znet)++ | S02-types/whatever.t
Test whatever regex curries

RT#131409: rt.perl.org/Ticket/Display.html?id=131409 Rakudo fix: github.com/rakudo/rakudo/commit/d80fc376d2
travis-ci Rakudo build failed. Zoffix Znet 'Fix whatever curry with regexes 12:38
travis-ci.org/rakudo/rakudo/builds/336515291 github.com/rakudo/rakudo/compare/d...0fc376d201
buggable [travis build above] ☠ Did not recognize some failures. Check results manually.
tbrowder hi, #perl-dev 15:17
did i see something here about nqp::elems being faster than +@array? ii that is a general rule i think there is a fair number of those in the master branch 15:21
at least 106 instances with a fast grep 15:27
moritz in rakudo or in the nqp sources? 15:32
jnthn I think the NQP optimizer rewrites many +@foo into nqp::elems(@foo) automatically 15:37
Geth nqp/master: 9 commits pushed by pmurias++ 16:02
travis-ci NQP build failed. pmurias '[js] Check for known encoding in codecs' 16:21
travis-ci.org/perl6/nqp/builds/336617768 github.com/perl6/nqp/compare/99493...48d45cf6a3
tbrowder moritz: 106 in nqp source in nqp repo 16:46
pmurias 17:49 < tbrowder> moritz: 106 in nqp source in nqp repo 17:14
sorry
github.com/perl6/roast/commit/74ea399d8e runs $*EXECUTABLE with 'env -' 17:15
which means that we run $*EXECUTABLE with an empty set of env variables
that causes problems for ./perl6-js as it takes node from $PATH 17:16
Geth roast: f9f5034a1e | (Aleks-Daniel Jakimenko-Aleksejev)++ | S32-exceptions/misc.t
Tests for RT#125674 (todo-ed)

There is a commit¹ with a message that claims that the issue is now resolved, but it does not seem to be the case.
  ¹ – github.com/rakudo/rakudo/commit/5747bc7121		 17:37
synopsebot RT#125674 [new]: rt.perl.org/Ticket/Display.html?id=125674 [LTA] error message prints wrong eject position (if True if { };)
pmurias jnthn: what are your thought about changing 'env -' into 'env' in roast? Is being able to run $*EXECUTABLE with an empty env something that we want to require/use in test? 19:08
jnthn: Exact line: github.com/perl6/roast/commit/74ea...93544e9R91 19:09
jnthn: it seems like an accident (as the interface to env is misleading) but it's something that is in 6.c 19:10
geekosaur you probably don't want the tests to be affected by random user environment config (TZ, LANG, LD_LIBRARY_PATH, etc.) 19:11
Geth rakudo: 95f23a5634 | (Zoffix Znet)++ | src/Perl6/Actions.nqp
Fix crashes with compile time evalution of `where` thunks

Phixes github.com/rakudo/rakudo/issues/1465
When we compile_in_context, we rake the .ann('outer')s to find all the good bits. However, the `where` thunks don't get that annotation and so the raking stops prematurely. This causes the thunk that tries to access some core symbols (like Junction ops) executed during being time explode.
Fix by adding annotating the thunk with the current pad.
roast: 79dff96fc9 | (Zoffix Znet)++ | S02-types/whatever.t
Cover crashes in compile-time `where` thunk evaluation

Closes github.com/rakudo/rakudo/issues/1465 Rakudo fix: github.com/rakudo/rakudo/commit/95f23a5634		 19:12
pmurias geekosaur: the user environment is passed in other cases it's just thrown away in this one 19:24
Geth roast: 195227f779 | (Aleks-Daniel Jakimenko-Aleksejev)++ | S32-io/pipe.t
Test that large blobs can be piped

Closes RT#129882. I am unable to reproduce the hang which is mentioned in the ticket, but this test *is* failing on all builds before 2017.06 with “got: '219264'”.		 19:30
synopsebot RT#129882 [open]: rt.perl.org/Ticket/Display.html?id=129882 [TESTNEEDED] [CONC] [IO] Proc with `.in.close` and `.out.slurp-rest` in different threads, hangs
Geth roast: 72f2c3984c | (Zoffix Znet)++ | MISC/bug-coverage-stress.t
Cover serialization crashes with traits + roles

RT#127959: rt.perl.org/Ticket/Display.html?id=127959		 19:34
synopsebot RT#127959 [open]: rt.perl.org/Ticket/Display.html?id=127959 [BUG] Missing serialize REPR function for REPR SCRef | When using a class using a trait that modify metamodel
travis-ci Rakudo build failed. Zoffix Znet 'Fix crashes with compile time evalution of `where` thunks 19:44
travis-ci.org/rakudo/rakudo/builds/336691463 github.com/rakudo/rakudo/compare/d...f23a5634d7
buggable [travis build above] ☠ Did not recognize some failures. Check results manually.
Geth roast: d266a4401a | (Zoffix Znet)++ | 2 files
Cover spesh regression with giant enums in packages

Closes github.com/rakudo/rakudo/issues/1413 MoarVM fix: github.com/MoarVM/MoarVM/commit/4733449165 MoarVM followup issue: github.com/MoarVM/MoarVM/issues/782		 19:50
roast: 469ebbc75b | (Zoffix Znet)++ | S02-types/array-shapes.t
Test Str can be used as indices of shaped arrays

Tests github.com/rakudo/rakudo/issues/1297		 19:57
roast: 85f55e2b23 | (Zoffix Znet)++ | S02-types/array-shapes.t
Add test description
 19:58
samcv going to add some roast tests for windows-1251 and windows-1252 now (currently we have tests for neither) 20:01
yoleaux 10:37Z <Zoffix> samcv: did you by chance miss pushing some commits? Travis says current Rakudo build fails to compile, yet master of all repos compiles just fine for me. I tried doing bumps, but there's a bunch of encoding-related failures in t/spec/S16-filehandles/io.t and a crash in t/spec/S32-str/encode.t with "Error encoding Windows-1252 string: could not encode codepoint 129"
samcv ah the test is wrong then 20:02
since 129 doesn't exist in that encoding
not sure why we have a test to write unmapped characters to the filehandle that's been set as not having those chars 20:10
every other decoder of windows-1252 will not accept codepoint 129 and will fail to decode. so i think we should act the same 20:13
because previously we could create files which are unreadable by anyone 20:14
Geth roast/coercers2.0: b77770dc4f | (Zoffix Znet)++ | S12-coercion/coercion-types.t
Fudge anything not already passing
 20:16
samcv ah and looks like the jvm doesn't pass those tests either from looking at the fudging ZofBot
Geth roast: ccc4edb8ed | (Zoffix Znet)++ | S12-coercion/coercion-types.t
Start coercer spec sketch
 20:23
roast: b77770dc4f | (Zoffix Znet)++ | S12-coercion/coercion-types.t
Fudge anything not already passing
roast: ee09f13914 | (Zoffix Znet)++ | S12-coercion/coercion-types.t
Add sketch of more comprehensive spec of DefiniteHOW coercers

This will likely need to be pruned before 6.d is finalized. In particular, the :U ↔ :D coercers look to be just mindless over-engineering with no proper usecase to cater to.
roast: e2dc8f0c7f | (Zoffix Znet)++ | S12-coercion/coercion-types.t
Add Issue number annotation to test

Closes github.com/rakudo/rakudo/issues/1361		 20:24
pmurias samcv: re windows-1252, aren't the tests for being able to read/write unmapped chars in 6.c? 20:25
samcv they may be. but the tests are still wrong unless someone can tell me why we actually want to create unreadable files
pmurias according to a google the unmapped chars are sometimes used by legacy apps 20:26
samcv hm
pmurias s/google search/
samcv i can change it. but i'm building rakudo jvm and seeing how that acts 20:27
at least perl 5 and iconv both don't allow codepoint 129
with binmode :encoding(windows-1252) on perl5 or just setting the source as windows-1252 on iconv 20:28
pmurias I haven't used windows-1252 myself in the past so I don't have intutions on how it was used in the past
it's legacy crap so it's only uses nowdays is to interact with old stuff 20:29
samcv looks like python also throws as an error too 20:31
timotimo i believe python encoders can be configured to do something else 20:49
ilmari m: "\c[128]".encode("windows-1252", :replacement) 20:51
camelia ( no output )
ilmari m: say "\c[128]".encode("windows-1252", :replacement)
camelia Blob[uint8]:0x<3f>
ilmari m: say "\c[128]".encode("windows-1252", :replacement("x")) 20:52
camelia Blob[uint8]:0x<78>
ilmari m: say "\c[128]".encode("windows-1252")
camelia Error encoding Windows-1252 string: could not encode codepoint 128
in block <unit> at <tmp> line 1
samcv pmurias: so jvm throws trying to encode unmapped characters, but it doesn't throw on decode 21:19
do you know what node-js does?
TimToady: maybe you have some thoughts on this too? 21:31
TimToady
.oO(windows-1252-c8)
 21:33
Geth nqp: d32d84d4ba | (Samantha McVey)++ | src/vm/jvm/runtime/org/perl6/nqp/runtime/Ops.java
[jvm] Add windows-1251 as an available encoding
 21:34
samcv TimToady: hahah
personally, i'd think that we should stick to the standard for windows-1252, but we could add an option to have a less strict version that passes invalid codepoints through. it's easily doable from the moarvm side at least. 21:35
and then jvm rejects encoding invalid, but passes through unmapped codepoints to the respective unicode control characters further complicating things 21:36
i could do the same thing on moarvm, but the question of whether that is a good idea is unsure to me 21:37
travis-ci NQP build passed. Samantha McVey '[jvm] Add windows-1251 as an available encoding' 21:51
travis-ci.org/perl6/nqp/builds/336739865 github.com/perl6/nqp/compare/b348d...2d84d4ba73
pmurias samcv: node.js decodes them as unicode replacement characters 22:19
samcv: I had to write my own windows-1252
samcv ah
so what is your opinion? should we just let the user decode/encode invalid files? 22:20
or not?
moritz would rather throw exceptions than silently produce wrong/incomplete answers 22:21
Geth roast: a7590d6543 | (Aleks-Daniel Jakimenko-Aleksejev)++ | S32-str/utf8-c8.t
Test malformed utf8 filenames roundtripping properly

Fixes RT#127671. Please feel free to object these tests (using similar concerns raised in the ticket). However, I still can't see how something in a subdirectory of /tmp can create a problem for any user. To me it sounds like nothing should ever touch or see these files anyway.		 22:23
synopsebot RT#127671 [open]: rt.perl.org/Ticket/Display.html?id=127671 [EXOTICTEST] ｢dir｣ dies if weird unicode sequences are encountered (dir;)
roast: 800bc0a1d9 | (Aleks-Daniel Jakimenko-Aleksejev)++ | S32-str/utf8-c8.t
Whoops. New bug?

Previous commit¹ had a thinko and was attempting to treat a file created in the previous test as a directory. This is now fixed, but it seems like there's another “Malformed UTF-8” issue.
  ¹ – github.com/perl6/roast/commit/a759...15e38ee713		 22:32
roast: 104173b332 | (Aleks-Daniel Jakimenko-Aleksejev)++ | S32-str/utf8-c8.t
Use make-temp-dir and make-temp-file

  irclog.perlgeek.de/perl6/2018-02-02#i_15771428		 23:04
japhb samcv, moritz: Being able to recover and work with partially-corrupted files (whether corrupted by previous broken software, by hardware failure, by stray cosmic rays, or by adverse interplanetary weather conditions) is useful. While strictness is definitely a fair default, there should always be an escape hatch for when your source data is broken and you still need to deal with it anyway. 23:09
samcv japhb: yeah i agree with that 23:10
i think it should be compliant by default and an option to relax it. but having the default be 'anything goes' seems like bad form
japhb I'm totally fine with compliant by default, relaxed by explicit option. 23:11
jnthn I think we should be able to configure decoders with a replacement char if you want non-explosiveness
japhb (self-clocking encodings)++ # For this very reason 23:12
samcv jnthn: what if you just want to pass it through?
jnthn I'm not sure what that'd mean
Well, aside from utf8-c8 style stricks
samcv well what it used to do before. 129.chr will map to 129
jnthn Which I guess is not utf8 specific
samcv for windows-1252
which is not in the charmap but we used to be able to write invalid windows-1252 files
jnthn D'oh 23:13
jnthn wonders if anyone ever wrote one ;)
japhb jnthn: Yeah, sometimes you want to map all corruption to a single replacement, and sometimes you want to say "reversibly map this to something that will return the same bytes when output again"
jnthn Yeah, if we support that then it should use the synthetics mechanism utf8-c8 does
japhb WFM
samcv jnthn: it doesn't really need to though 23:14
jnthn Because then it's just one special case in our internals
samcv since 129 exists in unicode
jnthn samcv: In that particular case, I guess so
japhb It would be good if we can avoid having to remember too many different special handlings of different encodings.
samcv i'm fine with changing it so we can either use 1. strict 2. replacement 3. loose (reversible as long as it can be encoded)
japhb I kinda like the idea of using synthetics as a general solution 23:15
samcv: Yeah, agreed
samcv though synthetics make sense if we can't map the input to unicode codepoints
jnthn Yeah. The thing is, Str is a Unicode grapheme-level representation, so it doesn't have any natural notion of "bytes to just cope with"
samcv but if it can map as an identical codepoint then i don't see an issue with that
jnthn Beyond the synthetics one we build 23:16
samcv for Shift-JIS synthetics may make sense but for one byte encodings we don't need to use synthetics in this case 23:17
japhb Do either of you have Uni I/O on your 2018 mental roadmap? 23:18
samcv now the other question is do we default to loose currently, and then go to strict by default with 6.d? or do we just go with strict now 23:20
jnthn samcv: I think except that encoding, we've been strict so far in general? 23:23
samcv jnthn: yes
though we have tests which test that unmappable characters can be decoded and encoded as windows-1252!
jnthn wat?! 23:24
grr
samcv yes
jnthn: also jvm can read unmapped characters but will not encode them
at least how we have it currently done on rakudo-j
jnthn If we do make it strict by default, then we should have implemented the way to ask for loose 23:25
samcv ok 23:26
jnthn So in the (probably unlikely, but still) event we break something, we can say "just add <option>"
(Or use it to have a different default, but that might be too much of an abundance of caution) 23:27
samcv jnthn: does gcc/clang detect that when we throw an exception, then code below it is unrecahable and optimized out? 23:29
jnthn Hmm...I think they may be marked with the "does not return" attribute, in which case it could use that 23:30
Whether it actually does, I'm not sure 23:31
japhb: Uni I/O - it's not particularly high on mine at the moment. Better support for working with binary data certainly is, however, and maybe some of the Uni I/O bits I could sneak in while doing that. But that still doesn't answer the hard questions, which are "what happens once I have a Uni" 23:32
(Besides "turn it into a Str") 23:33
But there's a load of open, and tricky, questions there
Not to mention "what happens if I match a Uni against a Regex"
japhb For me the big value of Uni I/O was "never block on looking for combiners" 23:35
jnthn Ah
japhb Frankly the option to do that one thing is enough for my particular use cases.
jnthn And then you're happy to immediately .Str it
japhb Yup. 23:36
jnthn OK, that's more realistic to expect this year
(I'd not quite realized the value that the I/O part alone could have.) 23:39
(So had it down as "not worth it until we've time to do the harder problem")
japhb Right, makes sense. 23:44
samcv i'm setting it up for now to have a #define to control wether it's strict or not 23:50
just a constant, so we can just swap that out with a variable to control it instead easily
jnthn Alright 23:51
samcv++
samcv jnthn: am i allowed to change the signature of the decode/encode op to add an option or do i have to make new ops 23:54
Geth nqp: a860fd3109 | (Samantha McVey)++ | tools/build/MOAR_REVISION
Bump MoarVM to revert throwing on windows-1252/1 decode errors

Changes: 2018.01-58-g8a3bcd9..2018.01-59-g6da2afe5a 6da2afe5a Default to not throw on windows-1251/2 decode/encode errors		 23:59
¦ nqp: version bump brought these changes: github.com/MoarVM/MoarVM/compare/2...g6da2afe5a

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!