eveo japhb: well, if you're on latest rakudo, see if the module works on bdf2019. That's before the proc refactor. 00:07
eveo goes to bed
japhb OK, I'll take a look 00:16
.tell eveo Yes, Inline::Python seemed to build fine at Rakudo bdf2019 (it was as you guessed broken at nom HEAD) 00:36
yoleaux japhb: I'll pass your message to eveo.
ugexe japhb: it fails based on the exit code 00:58
of running Build.pm
if you run zef with --debug it will show the command it uses to run the Build.pm, which you can run yourself without zef 00:59
windows failing since proc-using-proc-async merge ci.appveyor.com/project/moritz/rakudo/history 02:32
seems to run 20 tests, then gets dubious results on the remaining tests 02:35
trying to skip the tests and just install ends up failing during install-core-dist.pl with `Could not open (CompUnit::Repository::Staging). Failed to stat file: no such file or directory` (seems to think a CUR class name is a literal file path?) 03:27
gist.github.com/ugexe/fc2001addaa2...66c09c8373 03:31
wonder if its line endings messing up parsing of the precomp dependency files 04:17
[Tux] This is Rakudo version 2017.05-435-gaa9516be2 built on MoarVM version 2017.05-85-g21ee1a50 06:07
csv-ip5xs 2.850
test 13.233
test-t 4.368 - 4.370csv-parser 12.688
Geth rakudo/js: 81390d1818 | pmurias++ | 2 files
[js] Implement nqp::p6clearpre, nqp::p6inpre, nqp::p6setpre
 06:34
pmurias are there any plans to go through the tests that are not in spectest.data and check if they are valid? 07:12
for example I found a S04-phasers/exit-in-check.t which check that END blocks are called when we exit at CHECK time (and has a note that the it used to test the opposite behavior and was changed in 2006 audreyt to test for Perl 5 compatible) 07:16
as CHECK is compile time and END runtime it seems that exit in CHECK time should avoid END blocks 07:17
Geth roast: 51662b9399 | pmurias++ | 2 files
Remove old tests we don't run that seem broken

CHECK and BEGIN happen at compile time while END runs at run time. If we exit in compile time it doesn't seem to make sense to have
   runtime END block run.		 07:30
jnthn morning, #perl6 09:10
pmurias jnthn: morning 09:29
lizmat Files=1205, Tests=62036, 216 wallclock secs (13.18 usr 4.91 sys + 1315.71 cusr 130.26 csys = 1464.06 CPU) 09:34
Geth rakudo/nom: bd292225d4 | (Jonathan Worthington)++ | src/Perl6/Actions.nqp
Slightly more aggressive return handler check.

This makes more routines be compiled without the a return exception handler, which saves some instructions off their total code size.		 09:48
eveo japhb: so it's on Windows? Also, what was the output of the failure :/ 10:05
yoleaux 00:36Z <japhb> eveo: Yes, Inline::Python seemed to build fine at Rakudo bdf2019 (it was as you guessed broken at nom HEAD)
eveo buggable: speed 6
buggable eveo, ▂▁▁▁▁▁ data for 2017-06-12–2017-06-14; range: 4.341s–4.427s; 1% faster
eveo Actually now I remember Proc::Async being really buggy on Windows last month, so maybe that's why something's busted on windows 10:22
jnthn wonders why that'd happen, given in theory libuv abstracts away the differences :/ 10:23
That said, proc was also busted in various ways
eveo C:\Users\zoffi>perl6 -e "await Proc::Async.new($*EXECUTABLE, q|-e|, q|say 'hi'|).start" 10:25
===SORRY!===
Argument to "say" seems to be malformed
at -e:1
It's basically errors like that
jnthn :/
eveo ^ that's on 2017.04.3
jnthn Wonder if it's not sticking in quotes
Or some such
timotimo probably something like that
it has to cause the "say 'hi'" to become a single argument
it probably didn't succeed at that 10:26
there must be something like strace for windows where you can see what exact arguments it ended up passing
jnthn m: say 2.452 / 2.579 10:27
camelia 0.950756
Geth nqp: cabe421f02 | (Jonathan Worthington)++ | 2 files
Start conveying decont context in QAST -> MAST.

This means that we can, in various cases where we used to take a reference to a native lexical/attribute, now simply just get it boxed instead.		 10:32
rakudo/nom: 8ff980e780 | (Jonathan Worthington)++ | 2 files
Have decontrv op convey decont context.

Includes an NQP bump to get a QAST->MAST compiler with support for this.		 10:34
jnthn These two improve the code we get in consume-line-chars 10:35
And probably in a bunch of other places
(By making us not spit out code to make a lexical reference) 10:36
Amazingly, that makes my read lines and add up the chars benchmark go from 182 GC runs to just 41 10:38
timotimo jeez, that's pretty darn good 10:39
jnthn Wins 5% off total time 10:40
m: say 2.452 / 1.134 10:43
camelia 2.162257
jnthn Still some more opts to find before within a factor of 2 of Perl 5
tadzik damn, cool 10:44
jnthn The comparison is, fwiw 10:45
perl 6: my $fh = open "longfile"; my $chars = 0; for $fh.lines { $chars = $chars + .chars }; $fh.close; say $chars
perl 5: open my $fh, "<:encoding(UTF-8)", "longfile"; my $chars = 0; while ($_ = <$fh>) { chomp; $chars = $chars + length($_) }; close $fh; print "$chars\n"
pmurias do we have any potential of of passing a handler_out_of_dynamic_scope flag to a lexical_handler_not_found_error handler? 10:51
jnthn: it doesn't seem to be used current at all, but not calculating it would just shave a few C level memory assignments 10:56
Geth rakudo/nom: 7edf9da6b6 | (Jonathan Worthington)++ | src/vm/moar/ops/perl6_ops.c
Convey spesh facts better in p6decontrv.

This allows spesh, when it knows (or has guarded on) the return type already, to toss the return type checks entirely. This also boots out the Nil checking and error handling basic blocks, shortening the specialized bytecode by a couple of basic blocks, or approximately 12 instructions, which will in turn help bring more things under the inline limit.		 11:01
jnthn pmurias: I put that in so we could give better errors at some point in the future
Geth rakudo/nom: 14d7571311 | (Elizabeth Mattijsen)++ | t/spectest.data
Remove reference to tests removed from roast
 11:06
jnthn Lunch now. Later I will take a look at spesh of optional args, because we're making a pretty bad job of ripping out a lot of code we really should be able to. 11:07
I *think* that'll bring consume-line-chars well under the inline size limit
hah, and also we're doing an inlining of .defined, which is good, and we're realizing based on the argument type that its result is constant, which is good, but then still managing to spit out code that boxes an integer only to truth test it :P 11:11
anyway, bbiab :) 11:12
lizmat jnthn
jnthn++ :-)
nine jnthn: I bet that's much more fun than debugging concurrency issues ;) 11:14
jnthn back 12:14
nine: It is, unless I have to debug a spesh issue ;) 12:15
eveo m: class Foo { method foo { say $++ } }; Foo.new.foo; Foo.new.foo; 12:30
camelia 0
1
eveo A bit surprised by that... Sounds like using state vars in methods is setting yourself up for a lengthy bug hunt
moritz state vars do carry state around :-) 12:31
that shouldn't be a surprise, really
eveo I guess 12:32
moritz there are ordinary lexical variables and there are attributes
one for each desired behavior
eveo m: class Foo { method foo { -> { say $++ }() } }; Foo.new.foo; Foo.new.foo;
camelia 0
0
eveo I guess in this case it's just a new block each time
moritz right; closure cloning happens when the outer lexical scope is entered 12:33
closures are tricky. You may quote me on that :-)
eveo m: class Foo { method foo { (state $ = -> { say $++ })() } }; with Foo.new { .foo; .foo }; Foo.new.foo;
camelia 0
1
2
eveo moritz: you're good at closure cloning you say? Maybe fixing this bug will be easy for you: rt.perl.org/Ticket/Display.html?id...et-history 12:34
moritz eveo: I don't think so, but I can take a look 12:35
eveo \o/ 12:36
lizmat needs to get a different IP number 13:35
eveo starts a toast of HEAD on 64-core box 13:50
oh yeah, big guns \o/
dogbert17_ m: sub foo () {$ = 42}; for ^2000000 { $ = foo }; say now - INIT now # IOninja 13:55
camelia Cannot assign to an immutable value
in block <unit> at <tmp> line 1
eveo IOninnja what? 13:56
dogbert17_ a bug you reported, RT #130855 14:00
in case jnthn wants to get into some hardcore spesh debugging :)
eveo Oh yeah, now we're talking: perl6.party/assets/pics/toaster/htop.png 14:01
ugexe re Proc::Async quoting: I found this old note above a proc/proc::async abstraction - github.com/ugexe/zef/blob/8e74fbc5...m6#L37-L39 14:04
eveo bah, google killed my VM minutes before it finished the re-testing of failures. 14:42
This will have to do, I guess
Still had hangs, so 64-core vs 24-core didn't really make a difference. Took ~50m for entire run in both cases.
With a hack to count all the tests, even inside subtests, we have ~153374 tests (a bit inflateds, since stuff like `is_run` would count as more than one test) 14:44
A lot fewer than I expected
m: say "Only {153374-139011} tests are inside subtests"
camelia Only 14363 tests are inside subtests
eveo CPU utlization chart from the toast run: perl6.party/assets/pics/toaster/cp...zation.png 14:47
So ~10m to build moarvm/nqp/rakudo, 25m run to toast entire ecosystem. Then it waited for me to kill the hung install of one module, then another ~7m to re-test the ~250 failures 14:49
17 unsuccs: toast.perl6.party/ 14:54
eveo can't tell if #perl6 got really busy recently, or just a few people keep spamming eval commands... 14:57
I guess Building fail is what ugexe mentioned about exit code. 14:59
'cause run(:err, :out) always reported 0 for it
Also, looks like only 2 modules from previous toast are still burnt :) Good job \o/ 15:00
A few of new burns are due to failed spawned commands, so seems like the reason they were Succ in earlier toasts is just because of the run(:err, :out) exitcode bug 15:04
lizmat needs to get a different IP number again :-( 15:20
grrr this IP number is no good either :-( 15:25
mst lizmat: what problem are you having with IP addresses? 15:34
eveo: spot pricing?
lizmat mst: trying to get access to the n 15:35
NL name registry
it's been a while
mst and it hates certain IP addresses?
lizmat yeah 15:37
among other things
mst odd
lizmat I guess we missed a number of security upgrades
if it hadn't been for Xs4all no longer wanting to do our secondary DNS, I wouldn't be in this quagmire of rules and regulations :-( 15:38
eveo mst: spot? what spot? 15:40
mst eveo: you said google ate the VM when you hadn't finished, I was assuming it was something like an EC2 spot instance where you can get outbid
eveo mst: oh, google compute engine... Yeah, I'm using pre-emptive instances they can kill at any time in exchange for making them a lot cheaper
mst right, same principle then 15:41
ugexe bisect: my $proc = run "ls", :out, :err; my $result = ?$proc; say $result 16:06
bisectable6 ugexe, On both starting points (old=2015.12 new=14d7571) the exit code is 1 and the output is identical as well 16:07
ugexe, Output on both points: «run is disallowed in restricted setting␤ in sub restricted at src/RESTRICTED.setting line 1␤ in sub run at src/RESTRICTED.setting line 14␤ in block <unit> at /tmp/ysne1TuTOW line 1␤»
ugexe well anyways you can guess :) 16:08
eveo guess what? 16:09
ugexe something missed in Proc changes
eveo It now returns false?
Ah
ugexe it would get sunk before
eveo s: Proc, 'Bool', \()
SourceBaby eveo, Sauce is at github.com/rakudo/rakudo/blob/14d7.../Mu.pm#L96
eveo s: Proc.new, 'Bool', \() 16:10
SourceBaby eveo, Sauce is at github.com/rakudo/rakudo/blob/14d7...oc.pm#L151
eveo ugexe: sunk where?
at the end of run, I guess
Wait, no, I'm not following. 16:11
A sunk proc would explode.
ugexe yeah you're right, i thought there was more magic in that method
so probably need to add some awaits in methods that return exitcode related stuff? 16:16
or self.result.exitcode 16:33
eveo I don't get how this works now. 16:36
jnthn Before it used, oddly enough, close_i to do the blocking. Except it I think got stuff messed up when both :err and :out were specified
eveo Proc is in terms of Proc::Async, but Proc::Async is in terms of Proc?
jnthn hah
Proc::Async just uses a Proc instance to hold the exit code 16:37
eveo Here's Proc::Async returning a Proc and the Proc we're running gets its status set from Proc::ASync's proc: github.com/rakudo/rakudo/blob/nom/...oc.pm#L138
Ah
jnthn I'll have a look on my Windows box tonight at what's going on there 16:42
ugexe not just windows tho 16:43
I have a couple bug reports for zef that stem from `return ?$proc` returning false when everything worked - both on OSX and I can reproduce on linux 16:45
the windows thing might stem from that, but seems like its really just misparsing a line from what I can tell 16:46
eveo I'm looking into the return ?$proc thing now 16:48
jnthn Were :in/:out/:err used?
eveo And it looks like our current behaviour is correct.
jnthn eveo: As in, you think we were wrong before and that wrong behavior was relied on?
eveo jnthn: I think Proc.Bool needs to throw if it's called before all the opened pipes were closed. 16:49
jnthn Did it do that before?
eveo here, .Bool returns False because $!exitcode is still at -1 because pipes weren't closed. --> my $proc = run "ls", :out, :err; my $result = ?$proc; say $result
Before exitcode was 0 if you didn't close pipes, IIRC
jnthn argh why would you even test that before using the output? ;P 16:50
eveo And we had a bug report with people saying that it's wrong, but the real problem was their programs weren't closing the pipes, so I think the proper solution is to throw in such cases
ugexe because there used to be other bugs that worked around where closing .out before .err (or at all) would cause issues
jnthn :/ 16:51
eveo Oh, wait, default exitcode was -1 even before refactor
jnthn Yeah, I left the -1 in place
eveo Well, the attribute value, not what .exitcode returned
ugexe that also works as a silent process
eveo cpan@perlbuild4~/CPANPRC/rakudo (nom)$ ./perl6 -e 'my $proc = run "ls", :out, :err; .err.close, .out.close with $proc; my $result = ?$proc; say $result' 16:52
True
So yeah, it works fine now.
cpan@perlbuild4~/CPANPRC/rakudo (nom)$ ./perl6 -e 'my $proc = run "ls", :out, :err; .err.close with $proc; my $result = ?$proc; say $result' 16:53
True
hmmm... but the .out pipe is still open :\
eveo re-reads the code
Oh, nevermind. This build has my stupid patch in it. 16:54
jnthn heads home
bbl
ugexe my $proc = run "ls", :out, :err; say $proc.Bool; sleep 5; say $proc.Bool # this just doesn't feel right, despite not closing the pipes 16:55
for :in yeah i get it 16:57
eveo That gives False False for both for me. 16:58
I think we can inspect the promise when .sink/.Bool/.exitcode/.status are called and if it's finished, we can clean up the pipes and return right results.
Or not 16:59
Oh, we can leave the pipes as is and set the exit code
and status
ugexe that was my gut feeling
ugexe that zoffix would figure it out 17:00
eveo Gonna do that in ~5 hours, unless someone beats me to it
eveo &
maybe not even just inspect, but await even 17:01
eveo &&
ugexe `perl6 -e 'run "xxx", :out, :err;'` is another related behavior change 17:32
TimToady so of *course* someone shoots up the town just before TPC, again, sigh... 17:35
.oO(it's the Wild, Wild East all over again...)
 17:36
perlpilot what? /me needs to watch some news it seems
TimToady: At least they aren't shooting up the conference!
ugexe one of the other possible venues, milwaukee, also just had some shootings 17:38
eveo "again"? :) 17:51
Must be Python programmers X)
m: with "abcd" { say gather for 0.. .chars-1 -> $i { for $i+1 .. .chars -> $j { .substr($i, $j).take } } } 17:53
camelia (a ab abc abcd bc bcd bcd cd cd d)
eveo wonder if there's a shorter and nicer way to do that...
the "subsequences" in a string
m: with "abcd" { say gather for 0.. (.chars-1) -> $i { .substr($i, all $i .. .chars)».take } } 17:56
camelia ( a ab abc abcd b bc bcd bcd cd cd cd d d)
ugexe probably some way using array slices 18:01
perlpilot IF the order didn't matter, you could use .combinations :) 18:03
Though, the problem did make me wonder if there could be an enhancement to rotor that would take a coderef that would modify the steps as it went. 18:04
[Coke] eveo - do you want those dupes? 18:05
eveo gah, swap killed my box trying a way
perlpilot: I don't want combinations tho. I want contiguous subsequences
[Coke]: nope 18:06
jnthn Just pusehd a MoarVM patch to unbust precomp on Windows (it was indeed a bad arg quoting option) 18:07
eveo it don't matter much. I just saw Haskell has subsequences routine and wondered what it was
what it would be in p6
perlpilot I suppose you could construct a regex with :overlap that would do it. 18:08
(maybe) 18:09
b2gills m: say sort "abcd" ~~ m:ex/.+/ 18:10
camelia (｢a｣ ｢ab｣ ｢abc｣ ｢abcd｣ ｢b｣ ｢bc｣ ｢bcd｣ ｢c｣ ｢cd｣ ｢d｣)
perlpilot yeah, :exhaustive is better :)
eveo b2gills++
perlpilot b2gills++
though, you'd have to do some sort of transform if your characters weren't in lexicographic order 18:12
eveo What do you mean?
m: say "deadbeef" ~~ m:ex/.+/
camelia (｢deadbeef｣ ｢deadbee｣ ｢deadbe｣ ｢deadb｣ ｢dead｣ ｢dea｣ ｢de｣ ｢d｣ ｢eadbeef｣ ｢eadbee｣ ｢eadbe｣ ｢eadb｣ ｢ead｣ ｢ea｣ ｢e｣ ｢adbeef｣ ｢adbee｣ ｢adbe｣ ｢adb｣ ｢ad｣ ｢a｣ ｢dbeef｣ ｢dbee…
eveo Seems to work fine 18:13
b2gills m: say "abcd" ~~ m:ex/.+?/ # just make it non-greedy
camelia (｢a｣ ｢ab｣ ｢abc｣ ｢abcd｣ ｢b｣ ｢bc｣ ｢bcd｣ ｢c｣ ｢cd｣ ｢d｣)
perlpilot oh, so you don't care about the ordering of your subsequences
eveo Nah :) 18:14
perlpilot b2gills++ again.
I apparently don't even have my brain all the way on today
[Coke] ah, nice. I had gotten as far as: 18:20
<realizes that is giving the wrong ranges and backs away slowly> 18:21
dogbert17 [Coke]: I get a lot of strange spelling errors when I run make xtest in the docs repo. Is that normal? 18:22
xt/aspell.t ................ 2/309 # & ve 54 32: ver, vie, V, v, vec, veg, vet, Be, Ce, be, E, e, vex, VA, VI, Va, vi, vu, we, VD, VF, VG, VJ, VP, VT, Vt, vb, vm, vn, vs, vx, DE, Fe, GE, Ge, He, IE, Le, ME, Me, NE, Ne, OE, PE, Re, SE, Se, Te, Xe, ae, he, me, re, ye
xt/aspell.t ................ 6/309
# Failed test 'doc/Language/5to6-nutshell.pod6 has 1 spelling errors'
[Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] < $_[1]})}; a(4).map({"asdf".substr(|$_)}) 18:27
camelia ( no output )
[Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] < $_[1]})}; say a(4).map({"asdf".substr(|$_)}) 18:28
camelia (a as asd sd sdf df)
[Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] <= $_[1]})}; say a(4).map({"asdf".substr(|$_)})
camelia ( a as asd s sd sdf df df f)
[Coke] there. super complicated compared to the RE version. :)
dogbert17: no. there is only one failure at the moment, and it has a ticket associated with it.
ok 1 - doc/Language/5to6-nutshell.pod6 has no spelling errors 18:29
dogbert17 yeah, I made that commit, but I don't want to work on it until I understand why I get all these failures (you see one above)
[Coke] you have a local pending commit? put it in a branch or something, I can apply it locally and check. 18:30
dogbert17 I reverted the change to the pws file when I saw the failures, sadly that did not help. I must be missing something here 18:31
it seems to find errors which does not exist, very strange 18:32
interesting, if I run 'perl6 xt/aspell.t doc/Language/classtut.pod6' then no errors are found but if I run 'make xtest' then I get (among other things): 18:35
xt/aspell.t ................ 10/309 # & WHA 13 13: WHOA, WA, WHAM, WHAT, HA, WHO, WHY, YWHA, CHA, GHA, SHA, FHA, AHA
xt/aspell.t ................ 12/309
# Failed test 'doc/Language/classtut.pod6 has 1 spelling errors'
[Coke] what version of aspell do you have, and what dictionary? 18:41
dogbert17 @(#) International Ispell Version 3.1.20 (but really Aspell 0.60.7-20110707) 18:42
[Coke] I have Aspell 0.60.6.1 and aspell-dict-en (2017.01.22 from macports)
dogbert17 could it be /var/lib/dictionaries-common/aspell/aspell-en 18:43
dogbert17 has never really used this program 18:44
[Coke] so, you currently have no local changes, and perl6 xt/aspell.t is failing? 18:45
dogbert17 yes 18:46
[Coke]: gist.github.com/dogbert17/f65d88b8...06b8c2bbc5 18:51
[Coke] note that it's complaining that 'brac' isn't a work, but that's never a word in that file. 18:57
*word
dogbert17 yes, I saw that, strange indeed 19:00
geekosaur looks to me like it's dropping the last letter in all three cases 19:03
second is probably WHAT, third is brace, first... given it's 5to6nutshell, I'd guess vec? 19:04
so look for that word and the character immediately following
geekosaur wonders if the numbers are line and character 19:05
dogbert17 uses rakudobrew to switch back to 2016.12 and ..... 19:06
the errors are gone
dogbert17 errors out with 'This is Rakudo version 2017.05-439-g14d757131 built on MoarVM version 2017.05-85-g21ee1a504' 19:07
perhaps something for Zoffix to sink his teeth into 19:08
geekosaur ahah
eveo ?
dogbert17 eveo: backlog a little bit
geekosaur the first one is mishandling "you've"
eveo mehs 19:09
geekosaur so I will guess xtest is leaving out some suffix processing option
geekosaur did just notice a typo while looking at the second one 19:11
"Perl 5, so it will not be detail here. If you need to know the precedence and"
"detail " -> "detailed"?
AlexDaniel is this from 5to6 docs?
geekosaur github.com/perl6/doc/blob/master/d...p.pod6#L22 19:12
AlexDaniel if you read any of these documents from top to bottom, you'll probably see many other issues…
geekosaur the fact that I spotted that immediately suggests so :)
[Coke] dogbert17: I'm using 2017.05-315-g160de7e6f atm. 19:14
I'll upgrade later and try again. Also there's a doc ticket for something similar.
github.com/perl6/doc/issues/975
seems to be tied to a backslash there. 19:15
dogbert17 [Coke]: thx, will be interesting to hear what you find 19:18
perlpilot Is it a feature that the spell checker is checking code blocks too? (I would have made it skip code) 19:19
When I ran that test locally, I got failures in doc/Type/Supply.pod6 because of "GOOG" in a code example
[Coke] perlpilot: github.com/perl6/doc/issues/1375 19:27
perlpilot: yes, because there are comments in code.
it was a PITA, but now that have (HAD) a clean run, it's only stragglers like this that are an issue.
if someone wants to fix #1375, that'd be awesome.
and also, there's code in docs; class names, methods, etc. we have to whitelist those on the spellcheck anyway 19:28
dogbert17 the problem seems to occur when using :out from one Proc as :in for another proc. At that point the file sometimes gets cut short! 20:00
the last line of the cut 5to6-nutshell file looks like this '^A quick way to find the Perl 6 ve' 20:02
geekosaur glah. so what is copyng the data instead of dup-ing the fd? 20:03
dogbert17 geekosaur: the problem happens here: github.com/perl6/doc/blob/master/x...pell.t#L57 20:06
geekosaur no, it's triggered there. the actual problem is whatever is plumbing those together is doing it wrong
i.e. the problem is in the implementation of run 20:07
dogbert17 could be, sometimes it works and sometimes it fails
geekosaur right, it'll depend on how much is waiting to be read at that point 20:09
dogbert17 I more or less changed the 'run aspell' line to 'run cat' instead and saved the output of several runs in order to compare. 20:10
geekosaur it should be just plumbing the file handles together at low level (fcntl(F_DUPFD)), instead it's copying whatever data is already available and the rest is lost
dogbert17 geekosaur: does this shed any light on what's happening: gist.github.com/dogbert17/c41f5789...4c0d125d32 20:13
really afk &
geekosaur that shows it reading the current data and writing it to the other thing, yes 20:14
which is exactly the wrong thing to do when plumbing filehandles
dogbert17 geekosaur: is wrong because things can go wrong or for performance reasons 20:30
geekosaur because if you are connecting handles together, you want to connect the *handles*, not the data
say I spawn a long-running program and then want to connect its :out to another program's :in 20:31
I want to connect the handle so the other program can keep reading as the long-running program generates output
what it's actually doing, though, is copying whatever output happens to be available at the time you're connecting them, and writing it to the other program 20:32
which is fairly useless
so, what you want here is programA | programB
dogbert17 understood, thx for the explanation
geekosaur what run is doing is: programA >tmpfile & tail +0 tmpfile | programB 20:33
which will terminate as soon as tail hits the end of tmpfile, whatever it happens to be at that time
(note no -f on tail)
dogbert17 ahh
geekosaur anyway I would report this as a bug in the implementation of run 20:35
dogbert17 so in order for this to work, 'run' will have to be changed or [Coke] has to do some workaround in his code
geekosaur as I understand it, run is intended to work the way that code is using it (and as I described it) 20:36
it's jst not implemented correctly for the :in(filehandle) case
dogbert17 aha, I should check RT, perhaps this has already been reported 20:37
geekosaur which should F_DUPFD the handle's file descriptor to the new process's fd 0 instead of copying data
dogbert17 I see that MasterDuke has already made a report, rt.perl.org/Public/Bug/Display.html?id=129291 20:40
at least I think he describes the same problem 20:41
geekosaur no, that's a different problem
but it might be fixed as a side effect of fixing this, if the memory allocations in question are part of copying the data when it should be creating a pipe between the processes 20:42
japhb eveo: I'm not on Windows. Haven't run a Windows box in ages, actually. My problems were on Linux. 20:47
eveo: If you still need a dump of the Inline::Python errors, I can do a fresh build attempt.
dogbert17 a quick valgrind doesn't uncover anything suspicious 20:48
eveo japhb: nah, it's fine. We've identified one of the problems that's likely the culprit.
to be fixed soon
as in 1-2 hours
geekosaur I'm making a rakudobug about the run() thing 20:49
japhb eveo: Ah, thank you kindly. 20:50
dogbert17 geekosaur++ 20:51
japhb eveo: In perl6.party/assets/pics/toaster/htop.png why are some of the characters in '100.0%' red and others are yellow? 20:56
eveo waits for geekosaur rakudobug
geekosaur just hit send 20:57
eveo geekosaur: what run thing? the exitcode?
geekosaur no
eveo Ah, got it
geekosaur :in($proc.out) copies whatever is currently readable from $proc.out instead of building a pipeline
eveo Is this a regression due to Proc refactor or has this been that way for ages, do you know? 20:58
geekosaur which currently means aspell.t in the docs repo is prone to spurious errors as it gets whatever happens to be available at that point instead of the full ouytput
no idea
I would suspect it's always been that way, though, as the implementation would have been fairly obviously different 20:59
also my guess, from having poked at the moarvm <-> libuv interface before, is the current behavior is a placeholder because moarvm can't currently use the right libuv api 21:00
but that's just a guess, I don't know at what level things are being done wrong
jnthn Bah, can't damn win 21:08
On Windows the plugging stuff together at fd level didn't work at all either :/
Thus the current attempt at portability 21:09
The only thing I've learned about I/O is you can never do the right thing :P 21:10
japhb jnthn: System calls in general are tricky to get right. There are very particular reasons why the Unix APIs are the way they are, even though most of them seem quite weird at first glance. 21:13
jnthn japhb: Yeah, in this case I don't even really follow the issue though. In that, it's an asycnrhonous reader that, whenever it gets something, reacts by writing it to the input of the next thing. 21:14
japhb stdio buffering? 21:16
(I'm wondering if the current copying method races at eof/exit of input program) 21:18
jnthn Ah, a write/close race sounds more possible 21:19
geekosaur [14 20:31:08] <geekosaur> say I spawn a long-running program and then want to connect its :out to another program's :in 21:20
copying data at run() time of the second program will never handle this correctly
you would have to run() the first thing, capture all the output to a file, then run() the second thing with input from that file 21:21
there is no way to connect programs in a pipeline as currently implemented
jnthn geekosaur: I'm not disagreeing, I'm asking why. 21:22
geekosaur ?
how long are you going to wait and/or copy stuff to the new program?
[14 20:32:41] <geekosaur> so, what you want here is programA | programB
[14 20:33:09] <geekosaur> what run is doing is: programA >tmpfile & tail +0 tmpfile | programB
[14 20:33:36] <geekosaur> which will terminate as soon as tail hits the end of tmpfile, whatever it happens to be at that time
bugffering will skew this even more, but no amount of buffering aside from block-until-programA-exits will fix this if you are going to implement it as a copy operation 21:23
also I'd kinda expect perl 6 to be able to do pipes better than ms-dos 2 did 21:24
jnthn OK, my understanding of what run is doing doesn't match what you claim it's doing, so there's clearly something I don't understand correctly. 21:27
I don't think at this time of night I'm going to figure out what. Will re-read tomorrow.
geekosaur I hope run was not actually specced this way 21:28
MasterDuke m: my $p = run(:out, $*EXECUTABLE, "-e", "say qq|abc\n123|; sleep 1; say qq|def\n456|"); run(:out, :in($p.out), $*EXECUTABLE, "-e", q|say $*IN.read|).out.slurp(:close).say
camelia run is disallowed in restricted setting
in sub restricted at src/RESTRICTED.setting line 1
in sub run at src/RESTRICTED.setting line 14
in block <unit> at <tmp> line 1
geekosaur because the currene behavior is not what I expect if I tell it to connect handles together
jnthn *sigh* the whole Proc thing kinda just happened 21:29
geekosaur if it is specced, thoug, very well, we are ms-dos 2
I will wait for perl 6 to reinvent windows nt before expecting it to be able to create pipelines
so as I read the Proc docs, I still expect it to create a pipe 21:31
jnthn Yes, it's a reasonable expectation
ugexe before I think pipes might have worked on moar + linux, but thats it 21:34
definitely didnt work right on windows
geekosaur I think windows requires you to do things differently, yes
a pipe is a virtual filesystem object, and you can't just dup() stuff together like you do on unix 21:35
the first run() would have to know to allocate a pipe for the second one
if I understand this correctly, that is
ugexe right, but so ideally does it "do things differently" on windows to make it work "right", or is it so fundamentally different that windows should just do what it does now (wait for first proc to finish) 21:37
geekosaur goes digging
it's *not* waiting
that's part of the problem
the wait version would be slow and annoying but work
it's copying whatever happens to be available right then, and losing the resty
MasterDuke should what i tried above work? where work means print (essentially) the same thing as "abc\n123def456".encode
geekosaur MasterDuke, ideally it would. for that short an input, it might work as expected most of the time 21:38
but sometimes it'll be truncated
MasterDuke if i use $*IN.lines i get all the output of the first run. if i do $*IN.read i get `Failed to write bytes to filehandle: Broken pipe`
geekosaur yes, that's another thing that will happen now 21:39
MasterDuke and only the first part of the output, i.e., "abc\n123".encode
i'm just thinking we should have some good tests in roast of what should work/happen, but i'm not 100% myself on how things should be 21:41
geekosaur because tje second run does not wait for the first to finish, and the pipe is not connected between them so data would keep flowing; whatever data happens to be available for the parent perl6 to read at the time it performs the second run() is what goes to the second run(), and the first run() will get a write error because the parent perl6 stopped reading and exited after doing the second run()
eveo m: sub foo ($x) { LEAVE { CATCH { default { say 70; False } }; $x and die; True }; say 42}; dd foo 0 21:51
camelia WARNINGS for <tmp>:
Useless use of constant value True in sink context (line 1)
42
Bool::True
eveo would've thought LEAVE's value would be used for return. 21:52
geekosaur wonders if pipes not working on windows is a libuv bug 21:57
jnthn eveo: No, that wouldn't make sense given there can be multiple of them and you often use them to clean up 21:58
eveo Ah cool
geekosaur actually I am looking at the low level windows stuff in msdn and it looks like it should work, with one caveat 21:59
jnthn sub f() { my $fh = open "foo"; LEAVE close $fh; $fh.slurp } # boring example of when you'd really not want what LEAVE returns as the return value :)
geekosaur (windows file handles are not inherited by default like they are on unix, so you need to set the inherit flag for pipes to work for IPC like this)
but, this level of logic should be in libuv 22:00
eveo jnthn: when is @!post-spawn supposed to run? 22:01
in Proc it closes stdin
I mean a block is pushed onto it to close it 22:02
maybe in !await-if-last-handle if it's the last handle... 22:04
Ah, it already gets closed before the call. 22:05
Well, I'm gonna shove it to run to where we wait for the proc to finish running and see if any stresstests explode 22:06
eveo dines first
ugexe i dunno how valid this is, but gist.github.com/ugexe/118f826fa96e...3762dd1df6 sometimes shows the correct total and sometimes less 22:08
on the plus side it no longer deadlocks though :)
geekosaur "sometimes less" would be bad, but this should be a different issue 22:11
ugexe rt.perl.org/Public/Bug/Display.html?id=129882 # where i cribbed the example code from
jnthn eveo: After the $proc.start line 22:27
eveo jnthn: wouldn't that immediately close .stdin and prevent priting to .in? in `my $proc = run :in, "meows"; $proc.in.write: "foos"` 22:28
jnthn eveo: No, the :in case is handled in the elsif $in === True branch up above 22:29
geekosaur if this is what it sounds like, it's closing the parent's copy of the read side of the pipe for :in
which the parent does not need; it's for the child to read 22:30
the parent wants the write end
eveo jnthn: Ah, I see. Thanks.
jnthn And it only delgates the .close-stdin to the underlying Proc::Async when $proc.in.close is called 22:31
I'm utterly confused how ugexe's gist is losing anything, given on-write correctly `await`s also
Anyway, sleep time 22:35
'night
eveo night 22:37
ugexe well, it does only seem to happen when the buffer is above a certain size 22:41
so could be either the write or the read
eveo Don't see anything wrong with pipes that would make that code wrong. 22:53
Though found another rabbit hole:
$ ./perl6 -e 'with Proc::Async.new: «"$*EXECUTABLE" -e "say 42"» { (await .start).exitcode.say }'
42
1
BenGoldberg m: dd run("echo") 22:57
camelia run is disallowed in restricted setting
in sub restricted at src/RESTRICTED.setting line 1
in sub run at src/RESTRICTED.setting line 14
in block <unit> at <tmp> line 1
BenGoldberg m: dd Proc.new("echo 42") 22:58
camelia Proc is disallowed in restricted setting
in sub restricted at src/RESTRICTED.setting line 1
in method new at src/RESTRICTED.setting line 32
in block <unit> at <tmp> line 1
eveo It's pretty scare to have this huge breakage and not a single failing stresstest :/ 22:59
*scary
Which ultimately is the result of no-test coding. 23:00
Even with crappy TDD you cover most bases... like... exit code -_- 23:01
BenGoldberg I notice on <docs.perl6.org/type/Proc>, methods in, out, err aren't documented, except for the example code. They also don't show up in the searchbox. 23:05
I wonder how horrible it would be to add a new requirement to every rakudo class and method: Either it must be explicitly marked as internal, or it must have documentation. 23:06
geekosaur they're documented unhelpfully
"$in, $out and $err are..."
eveo "Unknown QAST node type NQPMu" ... I think I'm going the wrong way :)
geekosaur well, the documentation is helpful, just not findable
BenGoldberg If we generated docs from the code, then everything not marked as internal would have some sort of findable doc :) 23:08
eveo ugh this is all falling appart. 23:11
The whole Proc::Async returns a Proc, which is implemented in terms of Proc::Async is backfiring
BenGoldberg c: Proc::Async, 'start' 23:13
committable6 BenGoldberg, ¦Proc::Async: «Cannot find this revision (did you mean “ccd2bcc”?)» ¦: «Cannot find this revision (did you mean “all”?)»
BenGoldberg Undercover, Proc::Async, 'start' 23:14
eveo cover: Proc::Async, 'start' 23:16
Undercover: helpo 23:17
Undercover: help
stupid robot
Undercover eveo, Failed to figure out coverage status on perl6.WTF/SETTING__src_core_Proc_As....html#L210
eveo, Use cover: trigger with args to give to sourcery sub. e.g. cover: Int, 'base'. See modules.perl6.org/dist/CoreHackers::Sourcery
eveo This is unfun now 23:27
bastard just doesn't want to be awaited on :/ 23:28
ehehe. Called a method on the wrong thing and was catching exception about that 23:45
AlexDaniel “the problem seems to occur when using :out from one Proc as :in for another proc. At that point the file sometimes gets cut short!” 23:56
is it a new problem?
I think whateverables were suffering from something like this also…

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!