|
00:01
JRaspass left
00:03
sena_kun joined
00:05
Altai-man_ left
|
|||||||||||||||||||||||||||||||||||||||
| timotimo | so i've been playing around with lembark's workload and got heaptrack to actually output something helpful | 00:14 | |||||||||||||||||||||||||||||||||||||
| 120629739 calls to allocation functions with 88.41GB peak consumption from | |||||||||||||||||||||||||||||||||||||||
| [...] | |||||||||||||||||||||||||||||||||||||||
| 8783964 calls with 83.13GB peak consumption from: | |||||||||||||||||||||||||||||||||||||||
| uv__read | |||||||||||||||||||||||||||||||||||||||
| that looks like the program is either holding on to all the byte arrays that are being created by reading from the child process | 00:15 | ||||||||||||||||||||||||||||||||||||||
| or perhaps the child process is just writing as fast as it can go, which makes a bigger and bigger queue of read data pieces that we're too slow to process | 00:16 | ||||||||||||||||||||||||||||||||||||||
|
00:19
Voldenet left
00:20
Voldenet joined,
Voldenet left,
Voldenet joined
|
|||||||||||||||||||||||||||||||||||||||
| timotimo | well, i've sent sigstop to the zxcat process that's feeding data into the rakudo process and it's still going | 00:30 | |||||||||||||||||||||||||||||||||||||
| i haven't looked at the source of the workload program much yet, so i'm not sure what exactly the status output lines mean | 00:31 | ||||||||||||||||||||||||||||||||||||||
| the memory size is sloooowly growing, i'm not sure how the growth would have been during the same timeframe but without stopping the zxcat process | |||||||||||||||||||||||||||||||||||||||
| so maybe that's where i should look next | |||||||||||||||||||||||||||||||||||||||
| also i should make sure the request-garbage-collection isn't causing objects to be promoted to the old generation too fast and thus wasting gen2 space that won't "shrink back" | 00:34 | ||||||||||||||||||||||||||||||||||||||
| the input data is about 123 GiB uncompressed | 00:38 | ||||||||||||||||||||||||||||||||||||||
| so if rakudo is processing slow enough and xzcat is spitting out fast enough, i can imagine rakudo going over 128 GiB RSS quite quickly | 00:39 | ||||||||||||||||||||||||||||||||||||||
| oh bother i didn't "nohup" the task, and there's neither tmux nor screen installed either | 00:53 | ||||||||||||||||||||||||||||||||||||||
| it's consumed about 1.3k minutes of cpu time and it's just recently started writing out the "stubs" of the chunks, meaning it's finished "process-chunk" on a bunch of the threads and is now sorting hashes to write to the files | 00:58 | ||||||||||||||||||||||||||||||||||||||
| about 32k entries each per chunk it seems like | |||||||||||||||||||||||||||||||||||||||
| at this point top shows the RES to be 4.8 gigs | 01:00 | ||||||||||||||||||||||||||||||||||||||
| m: say 37.base(16) | 01:01 | ||||||||||||||||||||||||||||||||||||||
| camelia | 25 | ||||||||||||||||||||||||||||||||||||||
| timotimo | four more threads until all threads have passed on to stage "sort and output to file" | 01:02 | |||||||||||||||||||||||||||||||||||||
| m: my @pieces = ["a".."z"].roll(128) xx 33000; say @pieces.elems; my $start = now; @pieces .= sort; say now - $start | 01:03 | ||||||||||||||||||||||||||||||||||||||
| camelia | 33000 3.7065022 |
||||||||||||||||||||||||||||||||||||||
| timotimo | m: my @pieces = ["a".."z"].roll(128) xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ $^a.substr(0, 64) <=> $^b.substr(0, 64) or $a.substr(64, 64) cmp $b.substr(64, 64) }); say now - $start | 01:04 | |||||||||||||||||||||||||||||||||||||
| camelia | 33000 Cannot convert string to number: base-10 number must begin with valid digits or '.' in '3⏏5d s d m f l r w w j v j w i l c w i b i t p n w j v m l y g f m ' (indicated by ⏏) in block <unit> at <tmp> line 1 |
||||||||||||||||||||||||||||||||||||||
| timotimo | m: my @pieces = ["a".."z"].roll(128).join("") xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ $^a.substr(0, 64) <=> $^b.substr(0, 64) or :16[$a].substr(64, 64) cmp :16[$b].substr(64, 64) }); say now - $start | ||||||||||||||||||||||||||||||||||||||
| camelia | 33000 Cannot convert string to number: base-10 number must begin with valid digits or '.' in '3⏏5emmqkzndudltwxhkrilqjqjeffmroglztuwoozhjojyicscogaqdneakxgiemppl' (indicated by ⏏) in block <unit> at <tmp> line 1 |
||||||||||||||||||||||||||||||||||||||
| timotimo | oh, it's not even base 16 silly me | 01:05 | |||||||||||||||||||||||||||||||||||||
| m: my @pieces = [|"a".."f", |"0".."9"].roll(128).join("") xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ :16[$^a].substr(0, 64) <=> :16[$^b].substr(0, 64) or $a.substr(64, 64) cmp $b.substr(64, 64) }); say now - $start | |||||||||||||||||||||||||||||||||||||||
| camelia | Potential difficulties: To apply a Slip flattener to a range, parenthesize the whole range. (Or parenthesize the whole endpoint expression, if you meant that.) at <tmp>:1 ------> 3my @pieces = [7⏏5|"a".."f", |"0".."9"].rol… |
||||||||||||||||||||||||||||||||||||||
| timotimo | m: my @pieces = [|("a".."f"), |("0".."9")].roll(128).join("") xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ :16[$^a].substr(0, 64) <=> :16[$^b].substr(0, 64) or $a.substr(64, 64) cmp $b.substr(64, 64) }); say now - $start | 01:06 | |||||||||||||||||||||||||||||||||||||
| camelia | 33000 Cannot convert string to number: trailing characters after number in '03249⏏5b3469b21abd139d913e9b2607eeced66f19c56cc6c1fb6ed18858d5957ff62e138268847dbf8a7288a7ed55f186335db3e32bec36c3015d7ad5e823a31df7' (indicated by ⏏) in block <u… |
||||||||||||||||||||||||||||||||||||||
| timotimo | i'm using :16[] wrong, aren't i | ||||||||||||||||||||||||||||||||||||||
| m: my @pieces = [|("a".."f"), |("0".."9")].roll(128).join("") xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ parse-base($^a, 16).substr(0, 64) <=> parse-base($^b, 16).substr(0, 64) or $a.substr(64, 64) cmp $b.substr(64, 64) }); say now - $start | |||||||||||||||||||||||||||||||||||||||
| camelia | (timeout)33000 | ||||||||||||||||||||||||||||||||||||||
| timotimo | m: my @pieces = [|("a".."f"), |("0".."9")].roll(128).join("") xx 33000; say @pieces.elems; my $start = now; @pieces .= sort({ parse-base($^a.substr(0, 64), 16), parse-base($^a.substr(64, 64), 16) }); say now - $start | 01:07 | |||||||||||||||||||||||||||||||||||||
| camelia | 33000 3.8476713 |
01:08 | |||||||||||||||||||||||||||||||||||||
| timotimo | two-arg sort is slooooowwwww | ||||||||||||||||||||||||||||||||||||||
| i think i should really recommend lembark to change that | 01:09 | ||||||||||||||||||||||||||||||||||||||
| i sigcont'd xzcat and the rakudo process immediately started growing again, stopped it again and it stopped growing | 01:13 | ||||||||||||||||||||||||||||||||||||||
| how exactly is run(...).out implemented? | 01:16 | ||||||||||||||||||||||||||||||||||||||
| is it "just" a react/whenever that appends to some buffer or channel as soon as it gets data? | |||||||||||||||||||||||||||||||||||||||
| anyway it's about time for me to go to bed; none of the threads has finished the sorting bit yet | 01:18 | ||||||||||||||||||||||||||||||||||||||
|
01:23
JRaspass joined
|
|||||||||||||||||||||||||||||||||||||||
| jdv79 | timotimo: interesting | 01:36 | |||||||||||||||||||||||||||||||||||||
|
02:02
Altai-man_ joined
02:04
sena_kun left
02:43
AlexDaniel left
02:44
AlexDaniel joined
02:45
AlexDaniel left,
AlexDaniel joined
02:55
MasterDuke left
03:29
Kaiepi joined
04:03
sena_kun joined
04:05
Altai-man_ left
04:40
JRaspass left
05:01
nebuchadnezzar joined
06:02
Altai-man_ joined
06:04
sena_kun left
07:45
squashable6 left,
squashable6 joined
07:51
JJMerelo joined
08:03
sena_kun joined
08:05
Altai-man_ left
08:25
leont joined
|
|||||||||||||||||||||||||||||||||||||||
| lizmat | Files=1307, Tests=113021, 217 wallclock secs (28.62 usr 8.34 sys + 3026.18 cusr 286.18 csys = 3349.32 CPU) | 08:39 | |||||||||||||||||||||||||||||||||||||
| [Tux] |
|
09:00 | |||||||||||||||||||||||||||||||||||||
|
09:14
AlexDaniel left,
AlexDaniel joined
09:16
AlexDani` joined
09:19
AlexDaniel left
09:23
JJMerelo left
09:27
Xliff joined
09:43
JJMerelo joined
09:45
AlexDani` is now known as AlexDaniel,
AlexDaniel left,
AlexDaniel joined
10:02
Altai-man_ joined
10:05
sena_kun left
|
|||||||||||||||||||||||||||||||||||||||
| Geth | rakudo/remove-Seq-nonsense: c9a4f07de2 | (Aleks-Daniel Jakimenko-Aleksejev)++ | 2 files Remove caching of Seqs This is a temporary partial commit. If you see it on master then things went very wrong. |
10:44 | |||||||||||||||||||||||||||||||||||||
| rakudo/remove-Seq-nonsense: 9c19fddc47 | (Aleks-Daniel Jakimenko-Aleksejev)++ | src/core.c/Seq.pm6 Fix rakudo install Another temp commit. The method is skip-one, not skip. It used to build but not install, now both work fine. |
11:29 | ||||||||||||||||||||||||||||||||||||||
| rakudo/remove-Seq-nonsense: 95cadc8916 | (Aleks-Daniel Jakimenko-Aleksejev)++ | src/core.c/Seq.pm6 Fix a thinko in AT-POS It's the number of elements, so surely it's larger than the index. |
|||||||||||||||||||||||||||||||||||||||
| AlexDaniel | Altai-man_: can you run Blin on remove-Seq-nonsense branch? I'm expecting a bunch of stuff to fail, I just wanna know how much and how exactly it fails | 11:54 | |||||||||||||||||||||||||||||||||||||
| Altai-man_ | AlexDaniel, will run one tonight. | ||||||||||||||||||||||||||||||||||||||
| AlexDaniel | thanks | ||||||||||||||||||||||||||||||||||||||
|
12:03
sena_kun joined
12:05
Altai-man_ left
12:07
JJMerelo left
|
|||||||||||||||||||||||||||||||||||||||
| Geth | rakudo: f96a641fff | (Elizabeth Mattijsen)++ | src/core.c/Exception.pm6 Wordwrap X::Adverb message |
12:26 | |||||||||||||||||||||||||||||||||||||
| rakudo: 1f93724f71 | (Elizabeth Mattijsen)++ | src/core.c/Exception.pm6 Wordwrap X::Syntax::KeywordAsFunction message |
|||||||||||||||||||||||||||||||||||||||
| rakudo: 98c7e512a6 | (Elizabeth Mattijsen)++ | 2 files Re-imagine X::Syntax::ParentAsHash message Sadly, even though this error is a syntax error to be thrown at compile time, it sadly has no access to parser state so it cannot actually produce the line info. Try to make up for it as best as we can for now. |
|||||||||||||||||||||||||||||||||||||||
| rakudo/rakuast: 1cf5223bd4 | (Jonathan Worthington)++ | src/Raku/ast/regex.rakumod Compile regex sequences, alterations, conjunctions |
12:27 | ||||||||||||||||||||||||||||||||||||||
| lizmat | afk for a few hours& | 12:28 | |||||||||||||||||||||||||||||||||||||
| Geth | rakudo/rakuast: bd890a8813 | (Jonathan Worthington)++ | src/Raku/ast/regex.rakumod Fix copy-pasta; vendethiel++ |
13:53 | |||||||||||||||||||||||||||||||||||||
| rakudo/rakuast: 55e15dbfe4 | (Jonathan Worthington)++ | 3 files RakuAST hanlding of implicit $!, $/, and $_ At the compilation unit and routine level. |
|||||||||||||||||||||||||||||||||||||||
|
14:02
Altai-man_ joined
14:05
sena_kun left
|
|||||||||||||||||||||||||||||||||||||||
| Geth | rakudo/rakuast: 89921f7ea6 | (Jonathan Worthington)++ | 2 files RakuAST for regex anchors |
14:22 | |||||||||||||||||||||||||||||||||||||
|
14:36
softmoth joined
14:48
softmoth left
16:03
sena_kun joined
16:05
Altai-man_ left
|
|||||||||||||||||||||||||||||||||||||||
| moritz | I'm curious, why are things like RakuAST::SinkBoundary classes instead of roles? | 16:22 | |||||||||||||||||||||||||||||||||||||
| jnthn | moritz: 'cus the node to MOP call translator didn't learn roles yet | 16:23 | |||||||||||||||||||||||||||||||||||||
| moritz | ah, pity :D | ||||||||||||||||||||||||||||||||||||||
| jnthn | I'll get there eventually :) | ||||||||||||||||||||||||||||||||||||||
| It's listed as a "todo" in the README, even :) | 16:24 | ||||||||||||||||||||||||||||||||||||||
| Under the "tasks for anyone wanting to help" | |||||||||||||||||||||||||||||||||||||||
| moritz | who reads thoss, ever? :D | ||||||||||||||||||||||||||||||||||||||
| jnthn | Nobody crazy enough, apparently ;) | ||||||||||||||||||||||||||||||||||||||
| moritz | is that README in rakudo? didn't find it there | 16:25 | |||||||||||||||||||||||||||||||||||||
| jnthn | In src/Raku/ast | 16:26 | |||||||||||||||||||||||||||||||||||||
| moritz | ah | ||||||||||||||||||||||||||||||||||||||
| do I need any magic to build? | 16:30 | ||||||||||||||||||||||||||||||||||||||
| currently I get Only typename on RHS of constant implemented so far at line 5850, near "nqp::hash( | |||||||||||||||||||||||||||||||||||||||
| (after Configure.pl --gen-moar && make) | |||||||||||||||||||||||||||||||||||||||
| sena_kun for the record, has read that section, but is probably too dumb anyway | |||||||||||||||||||||||||||||||||||||||
|
16:34
squashable6 left
|
|||||||||||||||||||||||||||||||||||||||
| jnthn | moritz: Ah, maybe I need to bump NQP_REVISION...it needs an NQP with 30b1d064da767bb | 16:35 | |||||||||||||||||||||||||||||||||||||
| linkable6 | (2020-06-10) github.com/Raku/nqp/commit/30b1d064da Allow more interesting constants in NQP | ||||||||||||||||||||||||||||||||||||||
|
16:36
squashable6 joined
|
|||||||||||||||||||||||||||||||||||||||
| moritz | ok, I'll try with a newer nqp (and moar) | 16:36 | |||||||||||||||||||||||||||||||||||||
| Geth | rakudo/rakuast: f000b5780c | (Jonathan Worthington)++ | 2 files RakuAST for various built-in char classes |
16:38 | |||||||||||||||||||||||||||||||||||||
| rakudo/rakuast: 5469d3cf14 | (Jonathan Worthington)++ | tools/templates/NQP_REVISION Bump NQP_REVISION So that the rakuast branch builds "out of the box". |
|||||||||||||||||||||||||||||||||||||||
| rakudo/rakuast: 724c683fca | (Jonathan Worthington)++ | 2 files RakuAST support for basic regex quantifiers Along with backtrack modifiers. |
17:08 | ||||||||||||||||||||||||||||||||||||||
| jnthn | Almost enough to do 10-regex.t. It uses / <alpha> / which means I need to model assertion syntax too. But not right now...hometime. | 17:12 | |||||||||||||||||||||||||||||||||||||
|
17:40
AlexDaniel left,
AlexDaniel joined,
AlexDaniel left,
AlexDaniel joined
18:02
Altai-man_ joined
18:05
sena_kun left
18:34
MasterDuke joined
|
|||||||||||||||||||||||||||||||||||||||
| timotimo | hm, we don't have "int" constants, right? | 18:44 | |||||||||||||||||||||||||||||||||||||
| in nqp, i mean | |||||||||||||||||||||||||||||||||||||||
| lizmat | QRegex/Cursor.nqp has this comment in it: | 18:53 | |||||||||||||||||||||||||||||||||||||
| my $pass_mark := 1; # NQP has no constant table yet | |||||||||||||||||||||||||||||||||||||||
| so I guess that is correct | |||||||||||||||||||||||||||||||||||||||
| timotimo | the mast compiler has a bunch of "constants" too | ||||||||||||||||||||||||||||||||||||||
| Geth | rakudo/rakuast: 89a19d7d15 | (Moritz Lenz)++ | src/Raku/ast/README.md Fix typo |
18:55 | |||||||||||||||||||||||||||||||||||||
|
19:22
vrurg left
19:42
vrurg joined
|
|||||||||||||||||||||||||||||||||||||||
| jdv79 | timotimo: did you get anywhere excidint with lembark's stuff? | 19:53 | |||||||||||||||||||||||||||||||||||||
| *exciting | |||||||||||||||||||||||||||||||||||||||
|
20:04
sena_kun joined,
Altai-man_ left
|
|||||||||||||||||||||||||||||||||||||||
| timotimo | i haven't touched it again today | 20:15 | |||||||||||||||||||||||||||||||||||||
| Geth | rakudo: 248664c6c3 | (Elizabeth Mattijsen)++ | src/core.c/Exception.pm6 Wordwrap X::Placeholder::Block message And slightly reword it |
20:33 | |||||||||||||||||||||||||||||||||||||
| rakudo: 3bc4fa90ad | (Elizabeth Mattijsen)++ | 2 files Re-imagine X::Syntax::Variable::Twigil message By providing it with more information, and make it more palatable to readers that read English as a second language, by describing the oxymoron rather than just using the word. |
|||||||||||||||||||||||||||||||||||||||
| timotimo | lizmat: since you're currently already working on exception output, i'd like to toss this your way for a quick look: gist.github.com/egmontkob/eb114294...ck-example | 20:44 | |||||||||||||||||||||||||||||||||||||
| lizmat | looks interesting, doesn't work for me on MacOS, but then again, I'm not on Catalina | 20:46 | |||||||||||||||||||||||||||||||||||||
| so not bleeding edge | 20:47 | ||||||||||||||||||||||||||||||||||||||
| I guess it could be used for the default .WHY ? | |||||||||||||||||||||||||||||||||||||||
| m: say Int.WHY | |||||||||||||||||||||||||||||||||||||||
| camelia | No documentation available for type 'Int'. Perhaps it can be found at docs.raku.org/type/Int » | ||||||||||||||||||||||||||||||||||||||
| timotimo | hm that's already clickable | 20:53 | |||||||||||||||||||||||||||||||||||||
| lizmat | yeah, because of your IRC client, I presume? It's just text in the code, afaik | ||||||||||||||||||||||||||||||||||||||
| timotimo | nah, this irc client doesn't do that | 20:54 | |||||||||||||||||||||||||||||||||||||
| in fact, if it's word-wrapped i have to switch to its special "raw mode" so i can click or copypaste it in full | |||||||||||||||||||||||||||||||||||||||
| lizmat | "No documentation available for type '$name'. | 20:55 | |||||||||||||||||||||||||||||||||||||
| Perhaps it can be found at docs.raku.org/type/$name" | |||||||||||||||||||||||||||||||||||||||
| hehegithub.com/rakudo/rakudo/blob/mast...Mu.pm6#L86 # nothing magic there | 20:56 | ||||||||||||||||||||||||||||||||||||||
| github.com/rakudo/rakudo/blob/mast...Mu.pm6#L86 # rather | |||||||||||||||||||||||||||||||||||||||
| Geth | rakudo: 16d24a2127 | (Elizabeth Mattijsen)++ | src/core.c/Mu.pm6 Wordwrap Mu.WHY |
21:05 | |||||||||||||||||||||||||||||||||||||
|
21:23
finsternis joined
22:02
Altai-man_ joined
22:05
sena_kun left
|
|||||||||||||||||||||||||||||||||||||||
| AlexDaniel | Altai-man_: .tell me when it's ready | 22:36 | |||||||||||||||||||||||||||||||||||||
|
22:36
AlexDaniel left
22:48
tbrowder left
22:50
kawaii left
22:52
SmokeMachine left,
zostay left
22:53
chansen_ left
23:27
lizmat left
23:41
Xliff left
23:43
Geth left
23:44
Geth joined
23:46
Geth left
23:47
Geth joined
|
|||||||||||||||||||||||||||||||||||||||