| Geth | rakudo: 345fbf5a06 | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6 More Telemetry tweaks and fixes - make sure Telemetry object correctly roundtrip - no longer try to make sense of Telemetry:U - Telemetry:U, just die - add some more return types to add optimization opportunities |
12:01 | |
| lizmat | Files=1229, Tests=75833, 323 wallclock secs (14.74 usr 5.29 sys + 2171.38 cusr 214.53 csys = 2405.94 CPU) | ||
| Geth | rakudo: b80d486c7e | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6 Handle empty Telemetry reports better |
12:20 | |
| rakudo: ba49b34329 | (Elizabeth Mattijsen)++ | t/07-telemetry/01-basic.t Add some more basic Telemetry tests |
12:22 | ||
| nine actually uses a push mover. Though frankly his girlfriend does most of the mowing. | 12:26 | ||
| Geth | rakudo: fe799a9888 | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm Handle thread exhaustion more gracefully - leave more CPU to the threads that *are* doing stuff - add $!exhausted attribute to ThreadPoolScheduler - this gets set if creating a new thread failed - which then inhibits trying to add any more workers |
13:12 | |
| tbrowder | .tell Zoffixi added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary <en.wikipedia.org/wiki/Perl_6> | 13:13 | |
| yoleaux | tbrowder: I'll pass your message to Zoffixi. | ||
| tbrowder | .tell Zoffix | 13:14 | |
| yoleaux | tbrowder: I don't know what you want me to say to Zoffix. | ||
| tbrowder | .tell Zoffix i added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary | ||
| yoleaux | tbrowder: I'll pass your message to Zoffix. | ||
| timotimo | m: Bag.new.pick(1) | 13:57 | |
| camelia | (signal SEGV) | ||
| timotimo | oh? | ||
| ah, moarvm not bumped | 13:58 | ||
| travis-ci | Rakudo build failed. Elizabeth Mattijsen 'Add some more basic Telemetry tests' | 14:18 | |
| travis-ci.org/rakudo/rakudo/builds/300587018 github.com/rakudo/rakudo/compare/b...49b34329bb | |||
| buggable | [travis build above] ☠ Did not recognize some failures. Check results manually. | ||
| MasterDuke_ | /home/travis/build/rakudo/rakudo/install/bin/moar --libpath=src/vm/moar/stage0 src/vm/moar/stage0/nqp.moarvm --bootstrap --module-path=gen/moar/stage1 --setting=NULL --no-regex-lib --target=mbc --output=gen/moar/stage1/NQPCORE.setting.moarvm gen/moar/stage1/NQPCORE.setting | 14:21 | |
| Segmentation fault (core dumped) | |||
| only on one job | 14:22 | ||
| Geth | rakudo: ufobat++ created pull request #1246: RFC: IO::Path.resolve for windows 2nd attempt |
14:41 | |
| synopsebot | RAKUDO#1246 [open]: github.com/rakudo/rakudo/pull/1246 RFC: IO::Path.resolve for windows 2nd attempt | ||
| ugexe | Does fe799a assume that once exhausted there can -never- be another thread created? | 15:31 | |
| timotimo | yes | 15:35 | |
| ugexe | Why? Cannot the thread use go down at some point and thus allow more workers to spawn? | 15:36 | |
| timotimo | what do you mean by "thread use"? | 15:38 | |
| ugexe | Well, what does exhausted mean? And is it truely a state that cannot be changed once reached? | 15:40 | |
| timotimo | the common failure case is "too many open files" | ||
| ugexe | yes, but what if 100 threads suddenly complete their task and there are now more files that can be opened | 15:41 | |
| timotimo | then you don't need to add another thread because 100 threads just became free | ||
| ugexe | hmmm I’m not sure this adds up right to me but I can’t really say why so just ignore me | 15:42 | |
| timotimo | i can see why you're worried about this | ||
| adding more threads is one mechanism we use to dislodge deadlocks that could be solved by finishing up some more work tasks | 15:43 | ||
| in that situation the whole program (or maybe just big parts of it) are at a standstill anyway | |||
| ugexe | Wouldn’t hours manual thread creation be gimped by this? | 15:57 | |
| hours / bouts of | |||
| or rather the scheduler gimped by bouts of manual thread creation | |||
| timotimo | yeah, Don't Do That :) | 15:58 | |
| manual thread creation should hardly ever be used | |||
| ugexe | Yes but if you want to be smarter than the schedule you should be able to (without supplying a different scheduler) | 16:02 | |
| this just seems like some line that, once crossed in the program, will change its running characteristics for the remainder of its runtime. But this may be a case of I prefer the program blowing up and telling me it’s broke than trying to work around my situation | 16:05 | ||
| so a naive compliment to this might reset the exhausted flag if I successfully create a thread manually. | 16:06 | ||
| timotimo | that would require Thread to know about the Scheduler in question, that's weird | 16:12 | |
| i wouldn't be against just preventing threads from being created with exponential backoff timing | |||
| i suspect before this commit it would have tried to build new threads 100 times a second orwhatever the interval for the supervisor is | 16:14 | ||
| that's clearly Bad | |||
| bbl | |||
| ugexe | yes not a real solution, but more trying to explain the behavior i expect | 16:20 | |
| AlexDaniel | releasable6: next | 17:02 | |
| releasable6 | AlexDaniel, Next release in 7 days and ≈1 hour. No blockers. 0 out of 155 commits logged | ||
| AlexDaniel, Details: gist.github.com/d8161556014814006f...e5af051797 | |||
| lizmat | timotimo ugexe I'm thinking about only trying to have another try at creating a new Thread once a second (aka once every 100 supervisor loops) | 18:26 | |
| ugexe: please note that in the current setup, a Thread === a worker, *not* a task | 18:33 | ||
| once started, a Thread will never stop (in the current setup) | 18:34 | ||
| it will just block for another task to appear as soon as it runs out of tasks | |||
| Geth | rakudo: fe1f8632c1 | (Elizabeth Mattijsen)++ | src/core/Baggy.pm Fix segfault on Bag.new.pick(1) Although it would be interesting to see how that was possible to begin with. Problem was, it should have selected the empty iterator, but there was no check on an empty Baggy in there yet. This commit basically adds that. |
18:38 | |
| timotimo | lizmat: with a newer moarvm it doesn't segfault, instead it throws an exception | 18:40 | |
| jnthn | lizmat: Pretty sure timotimo fixed the segv | ||
| heh :) | |||
| lizmat | ah, ok, good | ||
| ugexe | i see. so is the goal to allow the workers to change to another type of worker? or does work stealing make this pointless? | 18:41 | |
| timotimo | stealing allows workers that are bored to steal from other workers that are overloaded | 18:43 | |
| i haven't looked at the code to see if it crosses worker types or not | |||
| but i think it does, at least in some directions | |||
| ugexe | otherwise I question what this means for the performance profile of an application when worker/affinity differences are caused by what happened in the past | ||
| e.g. i run the application one time and get 9/1, and another time I get 4/5 | 18:44 | ||
| lizmat | although this implementation is much more advanced than before | ||
| it's still pretty simple in its assumptions | |||
| ugexe | which could be problematic for anything long running | ||
| timotimo | your system should allow you to get far more than 10 threads | ||
| lizmat | 1. whenever a worker pool seems to need another worker, we add one | ||
| I've now added: | 18:45 | ||
| 2. if we've run out of low level system threads, don't bother trying to add them | |||
| jnthn | github.com/rakudo/rakudo/commit/fe799a9888 introduces a regression | 18:46 | |
| prod-affinity-workers should be run every time | |||
| And should not be inside of the loop | 18:47 | ||
| lizmat | but I didn't move them into the loop ? | ||
| they were always there ? | |||
| jnthn | It was moved into unless $!exhausted { | ||
| lizmat | that;'s not a loop ? | ||
| jnthn | github.com/rakudo/rakudo/commit/fe...ec119fR545 | ||
| oh shit | |||
| should not be inside of the *unless | |||
| Sorry | 18:48 | ||
| That was confusing | |||
| lizmat | you mean to say that you always need to prod ? | ||
| jnthn | Yes | ||
| lizmat | ok, will do | ||
| jnthn | prod doesn't start new threads | ||
| It just steals work from a stuck affinity queue into the general queue | |||
| To try and break deadlocks | |||
| lizmat | ok | ||
| still, my change was spectest clean :-) | 18:49 | ||
| jnthn | If there is a test that'd catch this, I think it'd be a stresstest | ||
| I also wonder a little if there's a more robust way to detect the failure to start thread exception than by text | |||
| lizmat | well, then it should throw something else than an X::AdHoc :-) | 18:50 | |
| jnthn | Yeah, we could wrap it up in Thread.start I guess :) | ||
| timotimo | maybe make the catch much tighter so the only thing it's catching for is ... yes | ||
| or that | 18:51 | ||
| jnthn | In the meantime, I'm meant to cook some dinner :) | ||
| timotimo | me bbl, too | ||
| lizmat | jnthn timotimo but if we wrap it in Thread.start, how will the supervisor every know ? | 18:52 | |
| jnthn | Thread.start is where the exception is coming from today, I think? | 18:53 | |
| Well, the nqp::startthread op inside of it, at least | |||
| timotimo | right, put a catch around only that, then throw a proper typed exception | ||
| jnthn | github.com/rakudo/rakudo/commit/fe...ec119fR280 | ||
| Yes, what timotimo said is what I meant by wrap | |||
| lizmat | ok, gotcha | 18:54 | |
| jnthn: I assume you mean nqp::newthread, right ? | 18:55 | ||
| jnthn | I forget the exact op names | ||
| There's one that creates the object and one that runs it | 18:56 | ||
| lizmat | yeah, newthread it is then :0) | ||
| $!vm_thread := nqp::newthread( | |||
| jnthn | github.com/rakudo/rakudo/blob/nom/...ead.pm#L40 | ||
| I think that one (nqp::threadrun) may be the one that throws the exception you're catching to day, rather than nqp::newthread | 18:57 | ||
| Not 100% certain though | |||
| lizmat | ok, will check | ||
| jnthn | But certainly a lot of the setup work actually happens in threadrun | ||
| And newthread just creates a handle object | |||
| Looking at the code, github.com/MoarVM/MoarVM/blob/mast...eads.c#L30 might throw | 18:58 | ||
| lizmat | so that would be nqp::newthread | 19:00 | |
| jnthn | Yup | ||
| Really cooking :) | |||
| bbl | |||
| Geth | rakudo: 57374490ae | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm Try creating a new Thread once a second if exhausted Instead of completely not try anymore. Also, always prod the affinity workers, even if thread creation will not work. |
||
| MasterDuke | AlexDaniel: you ran the toaster last release, right? have you done a new one for the upcoming release? | 19:14 | |
| AlexDaniel | MasterDuke: not yet | 19:28 | |
| MasterDuke: but see toast.perl6.party/ | 19:29 | ||
| MasterDuke: Zoffix++ did run it a few days ago | |||
| MasterDuke | ah, good | 19:30 | |
| AlexDaniel | MasterDuke: now I'm curious. Why? | 19:32 | |
| MasterDuke | just curious myself. lot fewer changes this month, hoped there was less fallout | 19:33 | |
| AlexDaniel | yeah, this release should be much quieter :) | 19:34 | |
| but we have a whole week to do disruptive changes! | 19:35 | ||
| Geth | rakudo: 14fbb5e76d | (Elizabeth Mattijsen)++ | 3 files Add typed error for thread creation exhaustion - and make sure the supervisor knows about it - bikeshedding on the new X::Exhausted exception may start |
19:42 | |
| ugexe | it should be X::hausted | 19:45 | |
| lizmat | and mauve of course :-) | 19:47 | |
| Zoffix | Don't see toast.perl6.party/ It's outdated. The change causing most of that fallout was reverted. | 19:52 | |
| yoleaux | 13:14Z <tbrowder> Zoffix: i added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary | ||
| bartolin | while looking for some failures on the jvm backend where a wrong multi candidate is selected, i stubled above this inconsistency: | 19:57 | |
| m: multi sub foo(Int $bar is rw) { }; foo(42) ## X::Multi::NoMatch | |||
| camelia | Cannot resolve caller foo(Int); the following candidates match the type but require mutable arguments: (Int $bar is rw) in block <unit> at <tmp> line 1 |
||
| bartolin | m: multi sub foo(int $bar is rw) { }; foo(42) ## X::TypeCheck::Argument+{X::Comp} | ||
| camelia | 5===SORRY!5=== Error while compiling <tmp> Calling foo(Int) will never work with any of these multi signatures: (int $bar is rw) at <tmp>:1 ------> 3multi sub foo(int $bar is rw) { }; 7⏏5foo(42) ## X::TypeCheck::Argument+{X::C |
||
| bartolin | with this change the first evaluation becomes a compile time error, too: github.com/usev6/rakudo/commit/db077336c5 | 19:58 | |
| but we have some tests that expect a X::Multi::NoMatch there. | |||
| i'd like to get some feedback whether the suggested patch makes sense. (at least it fixes a problem on the jvm) | 20:00 | ||
| MasterDuke | catching thing at compile time is good | 20:02 | |
| bartolin | the patch that catches the native case went in shortly before christmas: github.com/rakudo/rakudo/commit/c96deddc2a | 20:04 | |
| maybe jnthn can remember why the check for $literal was not applied for the non-native case ... | 20:06 | ||
| Geth | rakudo: 3e4ccce9b8 | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6 Tweak the T:I:Usage.preamble + allow util% in R_R_COLUMNS - also show final memory size for easy observation - make sure you can specify a column name with % in RAKUDO_REPORT_COLUMNS |
20:07 | |
| rakudo: 6959349e7f | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm Move variable definitions out of the supervisor loop - this does not seem to have a measurable effect on CPU - but saves about 3M of memory on a 10 second run - assuming this is not a leak, it's just about less GC churn - but we cannot tell since we cannot profile this cause its threading - wish there was a nqp::vmstat op for things like number of GC runs |
20:23 | ||
| lizmat | afk for some time& | 20:38 | |
| Geth | rakudo: 86e9f44aec | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6 Telemetry::Instrument.columns -> default-columns - this is a better name for what it returns - also add a .columns method that returns all columns in alphabetical order |
21:50 | |
| lizmat | good night, #perl6-dev! | 21:52 |