01:48 ilbot3 joined 02:07 _z left 03:21 geekosaur joined 03:37 vendethiel joined
masak good morning, #perl6 07:25
b2gills: if we had a .prime-factorize, should it return a Bag? :) 07:26
m: say bag("b", "a", "b", "b", "a").list
camelia rakudo-moar c8ec5a: OUTPUT«(a => 2 b => 3)␤»
masak hm
oh, this is #p6dev... :) 07:28
07:51 RabidGravy joined 08:40 btyler joined
RabidGravy "Stage mbc : Memory allocation failed; could not allocate 6386688 bytes 09:01
Makefile:434: recipe for target 'CORE.setting.moarvm' failed
make: *** [CORE.setting.moarvm] Error 1"
Boo
it worked a month ago 09:02
psch ...building rakudo?
09:04 brrt joined
RabidGravy yeah, albeit on a Raspberry Pi 09:04
psch ah, okay. guess moar went up in build time memory demand 09:05
RabidGravy stops all the things and tries again 09:06
jnthn MoarVM itself has mostly come down in memory use. But only a bit, and probably not enough to make up for growth in CORE.setting :) 09:14
RabidGravy this is a Pi 2 so the rev 1 is going to be utterly stuffed 09:16
stopping samba and icecast freed enough memory 09:30
11:42 brrt joined
stmuk hmmm pi 11:51
dalek ast: 2a715e5 | lizmat++ | S (2 files):
Adapt tests to changed Hash.gist behaviour

As implemented with 6a2ff75a61fdd49b55		 13:26
ast/6.c-errata: ca4aca8 | lizmat++ | S (2 files):
Adapt tests to changed Hash.gist behaviour

As implemented with 6a2ff75a61fdd49b55		 13:32
lizmat [Coke]: ^^^ These tests should maybe be marked todo ???
as they will now fail on vanilla 6.c / 2015.12 13:33
13:34 skids joined
jnthn lizmat: No, 2015.12 supports 6.c, not 6.c + errata :) 13:34
lizmat: And we don't truly version errata.
It's meant to be unconsequential enough that it doesn't matter
lizmat ok, so I've done the right thing ? 13:35
jnthn Yeah, looks like :)
lizmat ok *phew* :-)
jnthn errata is "what we wish 6.c had been if only we'd had a time machine" :)
lizmat fwiw, I think these tests have improved with this change
as more of the structure is visible in the test string
skids wonders what the performance hit of allowing Iterator.is-lazy to also return an Int for "I'm lazy but I know I'll produce this many more values" would likely be. 13:38
jnthn I worry more about the API complexity cost. 13:39
skids Well, any suggestions how to make the API actually handle those cases efficiently without adding complexity? 13:41
jnthn What does it handle inefficiently now? 13:42
skids Things like List.combinations(Range).elems need to iterate al combinations.
jnthn Or rather, why is how many values something marked lazy *can* produce interesting?
Isn't there a count-only that's used for that? 13:43
lizmat skids: but it doesn't, it the iterator supplies a "count-only" method
*if
skids count-only iterates.
Or he deafult does.
lizmat skids: it doesn't have to if there's an alternative
skids More importantly it exhausts.
lizmat yes, because the default has no choice
jnthn Indeed, but you can implement an iterator that does something smarter by implementing count-only 13:45
skids Well, it's docced to exhaust.
Which I consider a contract.
jnthn What is?
skids count-only
lizmat skids: e.g., the iterator for Str.comb() has: method count-only() { nqp::p6box_i($!pos = $!chars) }
this both exhausts (because it sets $!pos) and returns the number of chars immediately 13:46
skids: so, yes, it should exhaust, but that can be as easy as (re-)setting an attribute
skids I always took it that count-only was added so that .elems would not be used for the side effect of exhausting. 13:48
13:48 brrt joined
timotimo we've made this design so that users will have to explicitly ask for values from an iterator to be kept 13:48
skids Right but values != pre-known number of values. 13:49
jnthn .elems should ensure any side-effects are performed. 13:51
As should .count-only
lizmat jnthn: but caching is not a side-effect, right >
?
jnthn That doesn't mean you have to produce any values that are side-effect-free in order to answer count-only though 13:52
lizmat: Caching in what sense?
skids In that case what I'm looking for is to add the ability to know how many elems you have prior to performing side-effects, when possible, so I guess it would have to be an entirely new API addition. 13:53
lizmat if you numerify an iterator, it will cache the values produced by the iterator
sorry, Seq
skids ...which can be a lot of cache. 13:54
lizmat if you call .elems on a Seq, it won't cache
jnthn Right, .elems exhausts... 13:55
lizmat indeed, and it doesn't cache
jnthn So that @foo.grep(/.../).elems doesn't have to build a result list, for example 13:56
b2gills Perhaps there could be a role that states "this iterator knows how many values it produces, and requires reimplementing .count-only()" ( I have already tried this and I think it exposes a bug in List.Int(), or a bug in my own brain )
lizmat jnthn: indeed
jnthn b2gills: Why do we need a role, you just implement the method in your iterator, like all the "I know better than the default" methods to...
*do 13:57
lizmat m: my $a = <a b c d a>.grep(* eq "a"); say $a.elems; for @$a { .say }
camelia rakudo-moar c8ec5a: OUTPUT«2␤This Seq has already been iterated, and its values consumed␤(you might solve this by adding .cache on usages of the Seq, or␤by assigning the Seq into an array)␤ in block <unit> at /tmp/3gF0giOmjX line 1␤␤»
b2gills The role is to push more information to the compiler
lizmat m: my $a = <a b c d a>.grep(* eq "a"); say +$a; for @$a { .say }
camelia rakudo-moar c8ec5a: OUTPUT«2␤a␤a␤»
jnthn b2gills: But...what would it do with it?
lizmat b2gills: but you can already override all of the Iterator methods ??? 13:58
skids Another thing I considered was a LimitedIterator where you predeclare the number of elements, and it chops if you try to produce more, or pads with Nils if you do not produce enough, and whether that would be useful for infix:<Z> This could double for my purposes in the case where the limit matches reality.
lizmat skids: sounds like a useful role to have in Rakudo::Internals 13:59
jnthn Sounds more like a module to me.
b2gills There was some code that I was looking into that would work a little better if it had a way of knowing that method was re-implemented, but again it may not be the best way of changing it
jnthn No, introspection-based solutions are generally not a good way to go 14:00
b2gills I was thinking of using it for multi-method dispatch
skids Well, a module doesn't prevent combinations.elems from taking huge swaths of time/cache.
jnthn I still don't see why it has to *now* 14:01
Why can't it hand back an iterator that implements count-only? 14:02
skids It doesn't in the general case but the range case needs to build a map, and the map hardcodes it's Seq/Iterator, so there's no way to inject that without a full iterator re-implementation in that one case.
(map in my PR, GATHER currently) 14:03
It's the problem of having to do everything from scratch just to add a behavior tweak.
jnthn You can grab the iterator underlying a Seq, no? 14:04
So if you got the iterator back from map, you can just mix in a role that adds the count-only? 14:05
skids Sure and do what a run-time mixin?
jnthn Should work out fine
In theory if it's your map you can even get away with "does" to do it in place and return the very same wrapping Seq 14:06
skids I was under the impression that would be frowned on in core as hard to optimize.
jnthn Well, yes and no.
If you do mixins on a hot-path, it's a problem, sure
skids Well, we don;t know what the user is going to do with it in this case. 14:07
jnthn But it doesn't seem to me calls to combinations is the hot path, it's producing the combinations...
And there's no added cost when iterating the combinations
skids I was just thinking beyond this to more general cases, how to make that pre-knowlege useful in a composeable fashion. 14:08
jnthn Well, the other option that avoids mixins is just an iterator that delegates to the other, but is given a closure to run for count-only 14:09
timotimo composition instead of ... the other thing 14:10
skids Right and that begs the question of how smart we are going to get collapsing Iterator stacks in the optimizer.
Because if the answer is "very smart" then just wrapping it makes sense 14:11
moritz
.oO( ASAP = As Smart As Possible :-)
timotimo hm, how good is the "handles" trait? isn't it very good?
maybe we have to use the compiler services to make it the best it can be
moritz it's good enough that I used it on occasion 14:12
timotimo not sure how it's currently implemented
moritz: i meant in terms of raw overhead
jnthn Well, iterator chains are just method calls from one to the next
timotimo like, do we generate a little closure that makes sure the right thing gets called on the right object?
moritz timotimo: probably uses the same closure thingies that accessors used to
timotimo or do we somehow have something smarter than that?
OK, so a target for the compilation services
skids Still, it leaves me wondering whether the fact that the Seq/Iterator can non-exaustively tell you its elems would be useful to allow the querying of in the future. 14:13
Or some less wordy version of that sentence. 14:14
lizmat skids: but it can ?
skids How?
lizmat just create a "count-only" method that does what it is supposed to do without iterating 14:15
timotimo if the "count-only" method doesn't exhaust, that's fine
but how is the code that uses the iterator supposed to know it can do that?
skids RIght but how does an external user of the Seq know that that is the case.
lizmat the external user can never know this, not without looking inside 14:16
skids (before accidentally exhausting or caching)
They could if we had API for it.
timotimo .start-prevent-exhaustion, .end-prevent-exhaustion
lizmat this feels like a lot of overhead for the generic iterator case :-( 14:17
timotimo mhm
skids Or.. just have is-lazy return an Int in that case and then if you get a Bool you know, "nope"
lizmat I still don't see how this would generically work 14:19
away for 30 mins or so
skids Code that thinks it can do something useful with a pre-known .elems (like plan a hyper batch configuration, prealloc, whatnot) could 'given .is-lazy { when Int { do something useful } Bool { if $_ { do lazy stuff } else { do other stuff } }' 14:22
16:02 sortiz joined 16:07 vendethiel joined
lizmat fwiw, thinking about it more, I think I like the idea of an iterator being able to tell a client how many values it will produce without actually producing them 16:36
using "is-lazy" for this, feels hacky to me
I'm more thinking along the line of having a method "will-produce" 16:37
that would be implemented in the Iterator role as "method will-produce() { Nil }"
jnthn: ^^^ thoughts ? 16:41
Woodi but what it means "without producing" ? already produced&available without more work or total that can be produced from this point ? 16:59
jnthn lizmat: That name's certainly a bit off. May also want to consider ranges as well as an Int 17:26
lizmat: So grep for example could return ^$underlying-iterator's-prediction 17:27
lizmat: It may have some kind of use in batching for parallelism also
predict-elems or so might be closer name wise but probably still not right 17:28
lizmat Woodi jnthn : I envisioned this method to be called before any values have been produced 17:32
hence the "will-"
jnthn Yeah, I guess it puns badly with .produce 17:36
And doesn't really indicate a unit
It's too late for it now, though it's a slight pity count-only wasn't called elems-only...
lizmat but elems-only could be construed as needing to produce elems only 17:37
hence I named it "count-only" 17:38
jnthn But we do only need the number of elems? :) 17:39
lizmat iterators are really about a number of values, so to me that's self-explanatory 17:40
but I can also see value (pun intended) in a method that indicates: we're going to produce at least this many more values now 17:42
e.g. the IO::Handle.words Seq which reads in chunks, could perceivably do all the work on that chunk and return that number
jnthn I thought this was for asking for a prediction *before* iterating? 17:44
Asking how many things you've got "in the buffer" is the wrong way around
The design we have is pretty clearly about pushing requirements down the chain 17:45
Thus push-at-least
b2gills I think there should be some two way communication ( .is-lazy() for example ) 17:46
lizmat tries to wrap her head around racing an IO::Handle.words Seq
b2gills It might even be useful if a sequence knows it is sorted so that a .sort() ( that is with no arguments ) doesn't need to do any work. (as an example) 17:48
jnthn b2gills: Yes, for for static knowledge about an iterator, not dynamic.
b2gills: That sounds like one of those ideas that sounds great in theory but almost never crops up in practice... 17:49
lizmat: You'd want to race the operation you do on it, most probably
$foo.words.hyper.map(...) means that the parallelism is applying to the map
b2gills I didn't say it was a good example, just that it is sometimes a good idea for knowledge to pass both ways in a chain 17:50
lizmat but it would be nice if the hyper could takes chunks of words that sorta match chunks read from disk ?
jnthn lizmat: The thing implemenitng the hyper-ing can happily use push-at-least (if it isn't already) 17:51
lizmat
.oO( I guess I should take a closer look at HyperSeq first )
jnthn HyperSeq, .hyper, and .race are on my list of things I really need to look at also :)
What made it in was a reasonably sensible interface backed by proof of concept code :) 17:52
b2gills jnthn: That reminds me I meant to ask your opinion on this set of hyper/race map/grep tests github.com/perl6/roast/commit/5016...bfd657938f 17:57
lizmat m: use NativeCall; sub system (Str) is native {}; say system 'cowsay "what security?"' 17:58
camelia rakudo-moar c8ec5a: OUTPUT«sh: cowsay: command not found␤(Mu)␤»
jnthn The sleep doesn't inspire me much :)
b2gills I went for simple so it does rely on timing a little bit. I could have written it so that it is purely deterministic but that would rely on more features working the same
lizmat moritz: perhaps we should uninstall NativeCall on camelia ?
b2gills jnthn: to be fair it sometimes runs correctly without the sleeps 17:59
jnthn lizmat: It's been that way for ages and I don't think anyone's abused it...I'm more inclined to hope people will continue not to suck and make it so we can continue to try NativeCall things out ;) 18:00
b2gills: Then I'd remove them. Tests that depend on sleeping tend to be flappy
timotimo nobody needs to go through system via nativecall when you already can use all of nqp:: 18:01
jnthn b2gills: I don't think the sleeps are needed at all 18:02
b2gills: They look fine without them
b2gills I tested it many times, it does sometimes end up out of order 18:03
jnthn Yeah, but is that a test bug or an impl bug? :-)
Could easily be that the test is ok
b2gills I think it is purely that Rakudo+MoarVM isn't entirely deterministic
jnthn It doesn't have to be deterministic, it just has to give the correct results :) 18:04
b2gills I did have a version that used a start block, but it was very messy
jnthn Which for .hyper should be ordered
Doing it with promises seems sensible enough
b2gills I wanted a test that showed implementors that it should be out of order if .race was called
jnthn The hyper ones are OK 18:05
But the .race ones are bogus
b2gills So I tried to make it deterministic with non-deterministic features
jnthn They over-specify it
And, knowing a bit about how it's implemented, you'll not get those to pass reliably 18:06
And that's not a bug.
b2gills its like testing a random number generator
jnthn Pretty much
I'm not sure there's any value in trying to test that race doesn't preserve order
Just that it gives the right values in some order 18:07
b2gills It was meant more as an indicator to future implementations that it shouldn't even try to keep the order
perhaps just a sort afterwards ( and remove the sleeps ) 18:08
jnthn Yeah, for .race I'd sort afterwards
Typically the way these things are implemented is a "master" thread sending batches of work off to workers 18:09
And so it's very easy for the results to come back in a difference sequences than the work was completed. 18:10
b2gills well part of the point of the tests are that they are supposed to be implementation agnostic so it may be a good thing I don't know the details
jnthn Sure, but the .race ones assume too much of the details :) 18:12
b2gills If that's it I will start work on a PR
jnthn Sounds good 18:14
Thanks for adding some tests... :)
18:24 sortiz joined
b2gills done github.com/perl6/roast/pull/111 ( I so want to test that the first value isn't a 1, and the last isn't a 5 in the .race() tests ) 18:46
jnthn yeah but it legit could be sometimes :) 18:51
jnthn off to rest for today, but for those not following #moarvm also: the EVAL memory leak issue (and no doubt numerous other leaks) are fixed in HEAD, and I've done some GC tuning. Expcet to try another tuning patch tomorrow, then will bump HEAD. 19:18
And full write-up in my next post :) 19:19
dalek ast: 0eda7e0 | (Brad Gilbert)++ | S07-hyperrace/ (2 files):
Remove timing related stuff from hyper/race tests
 19:30
ast: 25649a8 | moritz++ | S07-hyperrace/ (2 files):
Merge pull request #111 from b2gills/reversed_hyper_race_2

Remove timing related stuff from hyper/race tests
kudo/nom: 2a04e13 | skids++ | src/core/List.pm:
Combinations fixes

   Fix sub combinations to return () not ((),) for absurd $k
   Fix List.combinations(Range) to use right values with careted Range endpoints
   Fix List.combinations(Range) to return () on an iterably-empty range
   Ensure only one empty list appears in result, only when a k==0 is involved
   Use a map/map instead of gather/take, no discernable performance impact
   ...someone may wish to weigh in on which will optimize better in future.
   ...this may help a plan to avoid pulling during .elems on the resulting Seq		 19:32
rakudo/nom: 0287290 | moritz++ | src/core/List.pm:
skids moritz++ 19:33
dalek ast: ab1da4f | (Christopher Bottoms)++ | S32-str/lines.t:
Added test for RT #126270
 19:35
ast: 38fdf08 | usev6++ | S04-statements/return.t:
Use 'run' to test for returning outside of routine

We can't test for 'Attempt to return outside of any Routine' with our toolchain from Test.pm or Test::Util.		 19:38
ast: 4832f2a | moritz++ | S04-statements/return.t:
Merge remote-tracking branch 'origin/return_outside_routine'
kudo/nom: ec52cce | skids++ | src/core/Any-iterable-methods.pm:
Fix several issues with List.squish

   Only evaluate :&as and :&with the minimum number of times
   Prevent IterationEnd from leaking into :&as and :&with calls
   Use intuitive ordering of :&with args for non-commutative :with
   Always pass :&with consecutive elements of the original list
   Once committed to doing a push-all, no need to mutate anymore
   Might possibly fix r-j RT#126527 -- needs someone to test
   Will commit roast tests for these behaviors after merge		 19:44
rakudo/nom: d17e3a1 | skids++ | src/core/Supply.pm:
ast: 0f70bbf | moritz++ | S32-str/lines.t:
Fix test plan
 19:56
psch skids++ 19:58
i'm gonna check ec52cce against r-j
20:31 travis-ci joined
travis-ci Rakudo build passed. Moritz Lenz 'Merge pull request #732 from skids/combos 20:31
travis-ci.org/rakudo/rakudo/builds/121249048 github.com/rakudo/rakudo/compare/c...87290dd114
20:31 travis-ci left 20:34 colomon joined 20:42 brrt joined 21:11 hankache joined 21:16 travis-ci joined
travis-ci Rakudo build passed. Moritz Lenz 'Merge pull request #685 from skids/squish 21:16
travis-ci.org/rakudo/rakudo/builds/121251835 github.com/rakudo/rakudo/compare/0...d231c5d139
21:16 travis-ci left
dalek ast: 0b3d846 | lizmat++ | S04-statements/return.t:
Unfudge now passing test
 21:29
21:33 colomon joined 22:16 skids joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!