🦋 Welcome to the MAIN() IRC channel of the Raku Programming Language (raku.org). Log available at irclogs.raku.org/raku/live.html . If you're a beginner, you can also check out the #raku-beginner channel!
Set by lizmat on 6 September 2022.
guifa FINALLY 02:00
all formatter patterns done
guifa الثلاثاء، ١١ أبريل ٢٠٢٤، ١٠:٣٢:٥٦ م التوقيت الصيفي الشرقي لأمريكا الشمالية 02:33
2024年4月11日火曜日 22時33分30秒 アメリカ東部夏時間
martes, 11 d’abril de 2024, 22:33:46 Hora braniega del este norteamericanu
guifa or for AntonAntonov вторник, 11 април 2024 г., 22:35:02 ч. Северноамериканско източно лятно часово време 02:35
now I just have to transfer over a few blocks of code tomorrow and I can release this sucker 02:36
snufkin @rf Thanks for your response, I did try this, but it seems to be overwritten when the HTML is produced. 05:17
I used this as the $response.html <html><head><meta charset="UTF-8"></head><body><h1>好</h1></body></html> This is what comes back: <html><head><meta http-equiv="Content-Type" content="text/html; charset=windows-1252"></head><body><h1>好</h1></body></html> 05:28
Nemokosch @snufkin what web server do you use? 06:53
snufkin Running hummingbird locally for development, it handles the serving. 07:56
Nemokosch I can only think of the server being messy... how else would it explicitly turn into Windows setup 08:00
snufkin It could be related to WSL perhaps. I'm running it in Ubuntu in WSL. 08:08
Nemokosch Issue or not, I wonder what actually "handles the serving" because I'm pretty sure Humming-Bird does no such thing 08:20
and the answer to that question would hopefully lead to an answer about the locale
snufkin > Humming-Bird has 2 simple layers, at its core we have Humming-Bird::HTTPServer which handles all of the low-level HTTP bits. Then you have the routing stack that handles: routing, middleware, error handling, cookies, etc. 08:25
It looks like RF has pushed a default to UTF-8 update to Humming-Bird so I'll try that when I am able. 08:26
Nemokosch oh it uses a vanilla socket 08:31
snufkin Do you know if that could explain the locale? 09:04
Nemokosch I don't know but the socket shouldn't be HTTP level, you'd think 09:05
and I don't see any obvious sign in the code that it would try to figure out a charset - prior to the recent modification that is 09:08
snufkin I don't know enough about networking/sockets to really know how it should/shouldn't be. 09:16
Voldenet snufkin: are you testing with curl or some other sane tool? 09:32
Voldenet $response.html writes `Content-Type: text/html` header, so whatever UA you're using, it could simply use that as a content-type instead of meta charset 09:40
Voldenet no other place attaches text/html to the response 09:40
snufkin @Volenet I'm testing with Chrome currently. Didn't use curl because intending it be a website. I can try it though 09:46
Voldenet Chrome is quite famous for being modern age internet explorer 10:09
for instance it tried to deprecate SMIL despite it being in active use 10:10
so I wouldn't be surprised if it rewrote the body of received html as well, sadly 10:11
snufkin Would Firefox be a better browser to test with? 10:15
Voldenet Since chrome is IE, you need to test on it for most common case 10:18
firefox is better at following set standards
snufkin Are all chrome derived browsers similar also? 10:24
Voldenet theoretically they could use patched version of chromium that revert risky changes, but I doubt they do 10:28
while I've not tested a lot of them, the ones I have installed do behave like chromium a lot 10:29
snufkin That makes sense. I'll see what happens with curl and firefox also. 10:38
[Coke] would it make sense to offer a single method to do a unique sort? either way you chain .unique/.sort you're processing the elements twice, and for long lists, a combined method (presumably an adverb on one of them) could do less work. 12:16
ugexe i think you could do that with the current sort by passing an appropriate callback 12:25
the problem with having a specialized method for that particular combo though is doing sort + almost any list mutating/filtering operation would do better with such a method, and that would be a lot of methods 12:27
Voldenet Those apis could produce iterators that would "know" about each other and then the method wouldn't be necessary 12:32
Voldenet using iterators there also allows implementing various optimizations in chained calls like `.unique.sort.kv.Hash` 12:50
lizmat .unique can be streaming, but .sort can not be, or am I missing something? 12:53
for a sort you would need to have seen all elements 12:54
Voldenet but iterators don't have to be streaming
iterators can evaluate result when needed
Voldenet so, .unique would not do anything, .sort would not do anything, only .Hash would consume all the iterators 12:54
so .sort would return UniqueSortIterator or SortIterator depending on the input 12:55
lizmat but that's how iterators already work ?
m: dd (2,3,1).sort
camelia (1, 2, 3).Seq
lizmat m: dd (2,3,1).sort.iterator
camelia Rakudo::Iterator::ReifiedListIterator.new
Voldenet m: dd (2,3,3,1).unique.sort.iterator 12:56
camelia Rakudo::Iterator::ReifiedListIterator.new
Voldenet m: dd (2,3,3,1).unique.sort.WHAT
camelia Seq
Voldenet m: dd (2,3,3,1).unique.WHAT
camelia Seq
lizmat m: dd (2,3,3,1).unique.iterator 12:57
camelia Any::Unique.new
Voldenet Ha, so the method for "unique sorting" can be fully internal
and no change is needed
lizmat except there's currently no way to indicate that iterator X is used as a source for iterator Y 12:58
that I know of, anyway
Voldenet It's not, but sort/unique can make the choice of right iterator 13:10
this feels like a hacky approach ix.io/4tgs 13:12
in fact iterable can be used so .iterator method can return right iterator when it's requested ix.io/4tgt 13:16
lizmat I'm not seeing the win here... .sort already effectively calls the iterator to create a list to work on 13:17
Voldenet it's related to earlier question whether making unique-sort() method makes sense 13:18
lizmat or do you want the sorting to be building the result list?
fwiw, such optimization might be better done at RakuAST level
when seeing a .unique.sort combo 13:19
Voldenet both approaches are not mutually exclusive
lizmat of course, this has it pitfalls as well, as we cannot be sure that the .unique and .sort would be the system's .unique / .sort as they are late bound
Nemokosch are these iterators meant to postpone execution as much as possible?
Voldenet Nope, which is pretty weird 13:20
lizmat docs.raku.org/type/Iterator.html 13:21
Voldenet my $x = 1..1000000; $x.sort; say now - BEGIN now
evalable6 1.129273468
Nemokosch WALK now 😄
lizmat iterators are meant to allow for lazy evaluation 13:22
without them, this would not be possible:
m: my @l = 1..Inf; say @l[100] 13:23
camelia 101
lizmat by accessing the 101st element, the list is reified upto that element
Nemokosch so here the idea is that actual evaluation could wait until we know how to do it better, no?
meta-laziness
Voldenet Yes, and in fact 13:24
m: my $x = 1..1000000; my $y = $x.iterator.sort; say $x.WHAT, $y.WHAT; say now - BEGIN now
camelia (Range)(Seq)
0.015011134
lizmat that sorts a list of 1 element containing an Iterator object ? 13:25
m: m: my $x = 1..1000000; my $y = $x.iterator.sort; say $x.WHAT, $y.WHAT; say $y 13:26
camelia (Range)(Seq)
(Rakudo::Iterator::IntRange.new)
Nemokosch I don't think "smh" translates into actually shaking your head well enough
Voldenet I'm not entirely sure why is magic `iterator` keyword even needed in this case 13:27
lizmat iterator is just a method ?? 13:27
Nemokosch Range probably has its nice list method
Voldenet s/keyword/word/
Nemokosch while Iterator probably does not
lizmat Iterator is a role 13:28
you use it to build iterator classes
the iterator method on a Seq returns the Iterator object it uses
Nemokosch the underlying iterator instance probably also doesn't provide list 13:28
Voldenet so Seq is Iterable 13:29
lizmat m: say Seq ~~ Iterable
camelia True
Nemokosch Range also is
Voldenet this means that .sort.unique could be more specialized for special cases of Iterables 13:30
lizmat sure... just write another .sort candidate
Nemokosch for Range, well... xD 13:31
99,9999% performance improvement
Voldenet m: my @x = (1..Inf).sort
camelia Cannot sort a lazy list
in block <unit> at <tmp> line 1
Voldenet I know this code is pure nonsense, but that's low-hanging fruit
lizmat how is that low hanging fruit ? 13:32
Nemokosch well yeah... by the same chance it does what it does currently, it could do the same thing but better... how many lines would that modification be? 13:33
lizmat how can you sort something without inspecting all of the elements ?
Nemokosch by knowing that it is ordered by definition
Voldenet Sort could inspect properties of iterable (like type) to not do any work
lizmat ok, well, then write a Range.sort
for the default case 13:34
if you pass a comparator, all deals are off
PR's welcome :-) 13:35
Nemokosch this reminds me of type checks
something that would allow you to define type transformations in any shape or form would be great
lizmat fwiw, the above would only work with either (1..Inf).sort or my @x := 1..Inf
Nemokosch to give a mundane example: if you have an odd number $x, $x + 1 is guaranteed to be an even number, no need to check for that 13:36
Voldenet or if you do (1..Inf).sum you should get -1/12 _obviously_ :P 13:37
Nemokosch 🧠
lizmat I guess an additional optional Iterator method could indicate that the elements are monotonically increasing
and sort could use that
and pass on the iterator as is in that case 13:38
Voldenet the comparator could also have specified ordering, but that'd probably require rakuast to work well
Nemokosch for the type checking - even something to say "trust me bro", like a kind of cast, could be good sometimes
Voldenet m: class A does Iterable { method iterator { (1..1000000).iterator } }; my $a = A.new; my $b = $a.sort; say now - BEGIN now 13:47
camelia 0.858667208
[Coke] sort - I liked the comment earlier about making it an optimization; .sort.unique could be optimized to .sort.squish 13:51
(and for now I'll just do that manually anyway!)
Voldenet Apparently Any.sort would need to be changed for different cases of self.iterator, it's doable 13:52
lizmat [Coke]: depending on the number of dupes, .unique.sort may still be faster 13:55
if you expect a lot of dupes, then .unique, if only a few, then .sort.squish I'd say 13:56
Voldenet $iterator.optimization-hint(:duplicate-probability(* > .5)).sort.unique 14:01
rf Morning folks 14:02
lizmat Voldenet: github.com/rakudo/rakudo/pull/5243 14:05
Voldenet Wow
Voldenet So with this change, Any.sort could skip work if self.iterator.is-monotonically-increasing 14:10
lizmat yup
Voldenet now that I think of it, it would break certain assumptions that expect sort to be always evaluated, but docs say: 14:14
Voldenet > method sort – Sorts iterables with cmp or given code object and returns a new Seq. 14:14
lizmat right 14:15
Voldenet I'm not sure if it implies that actual work must be done on Ranges
probably not
lizmat the PR did not break any spectests 14:15
adapted the PR to do the check in Any.sort, and removed the Range.sort candidate 14:17
Nemokosch why is that better? 14:19
Voldenet When your graph plotting sub and you provide xvalues, the plotting would obviously sort them first
:xvalues(1..100000) would skip sorting entirely 14:20
lizmat because it will apply the lazy pass-on behaviour on .sort to *any* object that has an iterator that is monotonically increasing, not just for Ranges?
Nemokosch fair enough, lol
Voldenet also, nice use case is that it's possible to sort lists only once 14:21
m: my @y = (1..100000); for ^10 { @y .= sort; say now - BEGIN now } 14:22
camelia 0.217464027
0.323814427
0.461667202
0.568463025
0.669587122
0.776436546
0.885371005
0.988860073
1.097826471
1.231786325
lizmat m: my @y = (1..100000); for ^10 { @y .= sort; say now - ENTER now } 14:23
camelia 0.12006733
0.113010976
0.105366429
0.14353936
0.117764756
0.103655504
0.111089322
0.115132678
0.105012741
0.159288894
Voldenet :)
either way, Seq can now know it is sorted 14:24
lizmat well, if the PR is merged, but yes 14:27
Voldenet the original point was that sort could be fully lazy so later operations can change how sort is being done 14:28
like .sort.head(10)
or .sort.squish
lizmat yeah, that would work, .sort.tail(10) not so much
eh... well, in the case of 1..* -) 14:29
but 1..10000 yeay
Voldenet that's up to later creativity of underlying Sequence 14:31
and since Sequences have `.eager` always, if someone really expected things to be eager, it can be forced
[Coke] in this PR, could calling .sort return a seq that has the flag set? 14:38
(or is that too much magic) 14:39
lizmat it does that... because it returns the underlying iterator in that case 14:40
so that magic is already there
[Coke] I mean if you have an unsorted list first, then sort it. 14:42
not if you start with a sorted list.
lizmat yeah, but I find Voldenets example of an argument to a sub that could be either sorted or not, so you call .sort on it 14:43
in that case: :points(1..10) would not need to be sorted at all
[Coke] finds an uncommitted change to App::Uni on an old dev box. 17:26
[Coke] (\_/) 17:39
(•_•)
> 🦋
^^ the very helpful change that lets you generate the bunny emoticon with a specified character. 17:40
[Coke] I think that was pre-pandemic. 17:55
tonyo heh, there used to be a dragon with a gold tooth in zef 18:51