01:39 jgaz left 02:55 hulk joined 02:57 kylese left 03:15 hulk left, kylese joined 04:25 Aedil joined
disbot2 <melezhik.> o/ 06:28
<melezhik.> I am hitting malformed UTF-8 exception when handling process stout in this line of code - github.com/melezhik/Sparrow6/blob/...kumod#L143 06:29
<melezhik.> I have googled the issue , tried few workarounds, nothing helped 06:30
<melezhik.> For example I tried pass :out, :bin to Proc.new 06:31
<melezhik.> I ended up piping command stdout ( before passing back to Raku ) to iconv -f UTF-8 -t UTF-8 -c ( -c silently drops none UTF symbols ) but this is a bit ugly as iconv side effect if buffering which I could not overcome 06:33
<melezhik.> Any ideas ? 06:34
<melezhik.> gist.github.com/melezhik/bac2eef77...1f1660d9b0 06:36
<melezhik.> Full log 06:37
<melezhik.> The issue occurs on the latest Rakudo version at least 06:41
<melezhik.> github.com/rakudo/rakudo/issues/6092 06:52
08:26 annamalai left 08:34 Sgeo left
disbot2 <melezhik.> The context : external command producing output passing to Raku is qemu, I run qemu vm which is bootstrapped via kickstart , I guess most of the output is valid text data , but a sone point it may have chucks of bytes / binary / none utf8 08:35
08:42 annamalai joined 09:39 abraxxa joined, abraxxa left, abraxxa joined 09:43 abraxxa left
lizmat try using utf8-c8 as encoding 10:21
11:08 sp1983 joined
sp1983 liz: do you mean this? - gist.github.com/melezhik/bac2eef77...nt-6044875 11:08
lizmat if Proc::Async.new takes an :enc argument, yes 11:09
sp1983 ok, this leads to this error during installation - gist.github.com/melezhik/bac2eef77...nt-6044876
hrm , probably the parameter should go first - $proc = Proc::Async.new(:enc<utf8-c8>,@arr); 11:11
let me try 11:12
at least this installs without errors
so :enc<utf8-c8> should not fall, even though data contain none UTF8 , right ? in my case I can't even control what goes from `qemu` bootstrap process stdout ... 11:14
lizmat looks like Proc::Async.new does *not* take a :enc argument 11:19
but yeah, that's the idea
sp1983 does not it? at lease this successfully installed and runs: 11:22
gist.github.com/melezhik/bac2eef77...nt-6044884
lizmat then the doc is incomplete 11:23
sp1983 ok, do I pass enc utf-c then to ProcAync then? or to it's methods in use ...? 11:25
do I -> how do I
11:27 sp1983 left 11:29 sp1983 joined 11:42 sp1983_ joined
sp1983_ oh, you mean then `$proc = Proc::Async.new(:enc<utf8-c8>,@arr);` is liable ? 11:42
just not documented ? 11:43
11:43 sp1983_ left
sp1983 liable -> correct 11:44
lizmat yeah 11:46
sp1983 ok 11:48
12:01 sp1983 left 12:41 sibl joined 13:08 sibl left 14:32 sp1983 joined 14:33 sp1983 left, sp1983 joined
sp1983 official documentation on ProcAsync.new reads that "multi method new(*@ ($path, *@args), :$w, :$enc, :$translate-nl, :$arg0, :$win-verbatim-args = False, :$pty, :$started = False --> Proc::Async:D)" 14:33
maybe this is a bug? as :$w, :$enc should go before `*@ ($path, *@args)`? 14:34
14:43 sp1983 left
ugexe seems easy enough to test no? 14:48
m: sub foo(*@ ($foo, *@bars), :$xxx) { say $xxx }; foo(1,2,3, :xxx(42))
camelia 42
ugexe m: sub foo(:$xxx, *@ ($foo, *@bars)) { say $xxx }; foo(1,2,3, :xxx(42))
camelia 42
ugexe regarding your actual issue, e.g. `whenever $proc.stdout(:enc<...>) { }` should work 14:51
15:09 sp1983 joined 15:14 silug left, silug joined 15:15 silug left 15:16 silug joined, sp1983 left 16:46 Sgeo joined
Voldenet lizmat: Proc::Async says that enc is supported in docs.raku.org/type/Proc/Async#method_new 17:10
m: my $proc = Proc::Async.new(<cat /dev/urandom>, :out, :enc("utf8")); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } } 17:13
camelia A react block:
in block <unit> at <tmp> line 1

Died because of the exception:
Malformed UTF-8 near bytes 31 4e f5
Voldenet m: my $proc = Proc::Async.new(<cat /dev/urandom>, :out, :enc("utf8-c8")); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } }
camelia ( no output )
Voldenet so I believe using :enc will work 17:14
the alternative could be using binary stdout entirely 17:23
m: my $proc = Proc::Async.new(<cat /dev/urandom>); react { whenever $proc.stdout(:bin) { .say; $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } }
camelia Buf[uint8]:0x<20 1E 22 37 8A 13 40 51 F5 48 4C 7D 30 C2 E7 50 2D 2C 74 15 0A A3 29 15 3A A9 C2 9C 5C 63 60 99 9C 4F EA 2A 12 61 F8 36 2E 9C FC D2 3C AA DE 5F E7 84 EF 15 D2 5D 68 DB E8 D5 39 1D 1E 59 E2 62 55 5A BE B8 B7 7A 13 1C 07 C6 B4 F7 98 DB 2D …
Voldenet you get Bufs, that way, not lines 17:24
though in case of CI this may be even useful to preserve binary output and render it into proper encoding on the fly
m: my $proc = Proc::Async.new(<cat /dev/urandom>); react { whenever $proc.stdout(:bin) { .decode("utf8-c8").ords.elems.say; .decode("windows1252").ords.elems.say; $proc.kill(SIGKILL) if $++ ~~ 5 }; whenever $proc.start { } } 17:29
camelia (timeout)287
128
558
256
1144
512
2408
1024
4745
2048
9525
4096
18870
8192
37680
16384
75681
32768
150513
65536
253224
109760
49147
21440
75793
32768
93290
40320
83746
36544
49101
21440
7…
Voldenet in fact, js could probably decode that binary better, because it can handle more encoding variants than raku at the moment 17:39
latin-2 for example 17:40
18:18 david7832 joined 18:28 sp1983 joined
sp1983 : my $proc = Proc::Async.new(:enc<utf8-c8>, <cat /dev/urandom>,); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } } 18:31
m: my $proc = Proc::Async.new(:enc<utf8-c8>, <cat /dev/urandom>,); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } } 18:32
camelia ( no output )
sp1983 my $proc = Proc::Async.new(:enc<utf8>, <cat /dev/urandom>,); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } }
m: my $proc = Proc::Async.new(:enc<utf8>, <cat /dev/urandom>,); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } } 18:33
camelia A react block:
in block <unit> at <tmp> line 1

Died because of the exception:
Malformed UTF-8 near bytes d3 d1
sp1983 m: my $proc = Proc::Async.new(:enc<utf8-c8>, <cat /dev/urandom>,); react { whenever $proc.stdout.lines() { $proc.kill(SIGKILL) if $++ ~~ 100 }; whenever $proc.start { } }
camelia ( no output )
sp1983 So looks like :enc could be the first argument as well 18:34
18:37 sp1983 left 18:39 sp1983 joined, sp1983 left 18:51 david7832 left
Voldenet Yes, you can put named arguments anywhere 19:44
m: sub foo(*@pos, :$named?) { say {:@pos, :$named} }; foo(1, 2); foo(:named, 1, 2); foo(1, 2, :named); foo(1, :named, 2)
camelia {named => (Any), pos => [1 2]}
{named => True, pos => [1 2]}
{named => True, pos => [1 2]}
{named => True, pos => [1 2]}
Voldenet even between positional arguments
20:09 Aedil left 20:20 annamalai left 21:08 librasteve_ joined
ugexe not before required positionals though 22:10
not sure why `*@ ($arg, *@rest)` doesn't count as a required positional though 22:12
m: sub foo(:$arg1, $foo) { }
camelia ===SORRY!=== Error while compiling <tmp>
Cannot put required parameter $foo after named parameters
at <tmp>:1
------> sub foo(:$arg1, $foo<HERE>) { }
expecting any of:
constraint
ugexe m: sub foo(:$arg1, *@ ($foo, @rest)) { } 22:13
camelia ( no output )
23:18 librasteve_ left
SmokeMachine m: sub a { my atomicint $a = 42; a => $a }; my :( atomicint :$a! is rw, | ) := a # is that a valid error (with LTA message) or should that work? 23:37
camelia ===SORRY!===
Cannot bind to QAST::Var resolving to a lexicalref
SmokeMachine github.com/rakudo/rakudo/issues/6094 23:42

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!