00:25 arkiuat left 00:33 arkiuat joined 00:34 wayland76 joined 01:32 arkiuat left 01:37 arkiuat joined
[Coke] Wikipedia's 429 HTTP error is telling us to wait 1000 seconds before the next request 03:27
03:32 librasteve_ left 04:35 arkiuat left 04:48 arkiuat joined 04:53 arkiuat left 05:14 arkiuat joined 05:19 arkiuat left 05:36 lizmat left 05:48 arkiuat joined 05:55 arkiuat left 06:32 arkiuat joined 06:37 arkiuat left 07:07 arkiuat joined 07:12 arkiuat left 07:34 arkiuat joined 07:39 arkiuat left 08:07 arkiuat joined 08:12 arkiuat left 08:34 arkiuat joined 08:39 arkiuat left 09:02 arkiuat joined 09:07 arkiuat left 09:35 arkiuat joined 09:40 arkiuat left 10:07 arkiuat joined 10:12 arkiuat left 10:33 lizmat joined 10:34 arkiuat joined 10:38 arkiuat left 10:55 arkiuat joined 11:06 arkiuat left 11:36 arkiuat joined 11:42 arkiuat left 12:01 arkiuat joined 12:06 arkiuat left 12:20 arkiuat joined 12:24 wayland joined, wayland76 left 12:29 arkiuat left 12:41 arkiuat joined 12:46 arkiuat left
[Coke] guessing someone did some nasty crawling with Cro and they soft-blocked that useragent. 12:51
lizmat does changing the user agent help ? 12:53
otoh, I think WP is protecting itself more aggressively against scrapers in general
[Coke] Yes, changed it to something like 'raku docs validator', no 429s 12:57
now back to just getting them on raku.land. :)
updating the sleep between 429s to at least 5, no more than 42s. 12:58
Geth doc/main: 5 commits pushed by (Will Coleda)++ 13:05
[Coke] That run hung, it seemed, but did find one issue, at least.
13:13 arkiuat joined 13:18 arkiuat left
Geth doc/main: fc938e0068 | (Will Coleda)++ | doc/Language/unicode_entry.rakudoc
A var, not the constant.
13:28
doc/main: a1920bde50 | (Will Coleda)++ | xt/rakudoc-l.rakutest
Once verified, cache that good result for the run
[Coke] weird, it's hanging again, just after checking ok 33 - HTTP 200 en.wikipedia.org/wiki/Greek_alphabet \#Greek_in_Unicode in # Subtest: doc/Language/unicode_entry.rakudoc 13:29
Same place as before.
using 0.7% cpu
13:32 arkiuat joined 13:36 arkiuat left 13:48 arkiuat joined 13:53 arkiuat left
[Coke] bug: www.gitbook.com/book/kyclark/metag...cs/details is a 200 but it's just showing a generic page. 14:01
14:19 arkiuat joined 14:27 arkiuat left
Geth doc/main: 362657f8ba | (Will Coleda)++ | 2 files
Remove link to defunct book
14:37
[Coke] not ok 67 - HTTP 503 github.com/MoarVM/MoarVM/blob/mast...ytecode.md 14:41
(that page seems fine, surprised we got a 503 from github) 14:42
14:58 arkiuat joined
[Coke] would be nice if we queued up all the https tests, sorted them by hostname, and ran multiple threads at once. 15:01
(so that we could have one thread blocking on 429s from github or whatever, while all the one offs can fire off one right after the other
15:08 arkiuat left
[Coke] had a half dozen links report broken but clicking on them manually they came up fine 15:16
(ah, they were all connection reset by peer or timeouts of some kind) 15:17
15:24 Guest5968 joined, Guest5968 left, Guest5968 joined
jubilatious1_98524 kyclark.gitbooks.io/metagenomics/content/ 15:34
[Coke] If someone wants it back, please submit a PR with the updated URL. :) 15:37
jubilatious1_98524++
15:56 arkiuat joined 15:59 librasteve_ joined 16:01 arkiuat left 16:20 arkiuat joined 16:25 arkiuat left 16:39 Guest5968 left 16:44 Guest4085 joined 16:52 arkiuat joined 16:57 arkiuat left 17:09 Guest4085 left 17:21 arkiuat joined 17:27 arkiuat left 17:56 arkiuat joined 17:59 Guest4085 joined 18:01 arkiuat left 18:23 arkiuat joined 18:28 arkiuat left 18:49 librasteve_ left 18:56 arkiuat joined 19:01 arkiuat left 19:23 arkiuat joined 19:28 arkiuat left 19:36 arkiuat joined 19:41 arkiuat left 20:02 Guest4085 left 20:08 arkiuat joined
Geth doc/main: 01ccea9186 | (Will Coleda)++ | 2 files
Revert "Remove link to defunct book"

This reverts commit 362657f8bacb21ccf0eb5429b5602e70b1f0d4e7.
20:18
doc/main: 988a2c874f | (Will Coleda)++ | doc/Language/faq.rakudoc
Update link to book

jubilatious1_98524++
jubilatious1_98524 Thanks coke! 20:22
[Coke] Sure, thanks for finding it! 20:28
20:34 arkiuat left 20:54 arkiuat joined 20:59 arkiuat left 21:26 arkiuat joined 21:30 arkiuat left 21:40 Voldenet left 21:41 Voldenet joined 21:52 wayland left 21:59 arkiuat joined 22:04 arkiuat left 22:25 arkiuat joined 22:30 arkiuat left 22:47 arkiuat joined 22:54 arkiuat left 23:07 arkiuat joined 23:15 arkiuat left 23:28 arkiuat joined 23:49 Guest4085 joined 23:50 Guest4085 left, Guest4085 joined, Guest4085 is now known as ds7832