🦋 Welcome to the IRC channel of the core developers of the Raku Programming Language (raku.org #rakulang). This channel is logged for the purpose of history keeping about its development | evalbot usage: 'm: say 3;' or /msg camelia m: ... | Logs can be inspected at colabti.org/irclogger/irclogger_log/raku-dev | For MoarVM see #moarvm
Set by lizmat on 26 April 2021.
00:02 reportable6 left 00:05 reportable6 joined 01:05 benchable6 left, nativecallable6 left, linkable6 left, sourceable6 left, evalable6 left, quotable6 left, coverable6 left, greppable6 left, notable6 left, unicodable6 left, statisfiable6 left, committable6 left, bisectable6 left, squashable6 left, tellable6 left, releasable6 left, bloatable6 left, shareable6 left, statisfiable6 joined, coverable6 joined, bisectable6 joined 01:06 notable6 joined, linkable6 joined, nativecallable6 joined, evalable6 joined, tellable6 joined, unicodable6 joined 01:07 releasable6 joined, shareable6 joined, benchable6 joined, committable6 joined, quotable6 joined 01:08 squashable6 joined, bloatable6 joined, sourceable6 joined, greppable6 joined 01:18 gfldex left, gfldex joined 01:44 kvw_5_ joined 01:47 kvw_5 left 01:52 sxmx1 left, Voldenet left, donaldh left 01:53 sxmx1 joined, Voldenet joined, donaldh joined, tobs joined, tailgate joined, jraspass joined, lucs joined 03:05 lucasb left 03:57 frost-lab joined 05:32 b2gills left 05:33 b2gills joined 05:54 geth_ joined 05:57 Geth left 06:02 reportable6 left 06:03 reportable6 joined 06:51 domidumont joined 07:57 dogbert17 left 08:03 dogbert17 joined 08:17 sena_kun left 08:18 sena_kun joined 08:43 geth_ left, Geth joined 09:28 Xliff joined
Xliff \o 09:30
Can someone look into R4356? 09:31
Or tell me how to track number of files Rakudo has open?
Is there a way this can be done with --profile or would --tracing be better? 09:32
MasterDuke or maybe something like `strace -e open,close -o files.log` and counting them in the log file? 09:39
--profile won't do it, --tracing might help. i'm not sure, but timotimo would know better 09:40
Xliff MasterDuke: What would you suggest I do (except golf... this one is too complex, I think) to get 4356 looked at? 09:52
I'm creating a docker image that (I hope) will have everything ready exept for the final step. 09:54
timotimo i'm not sure what exactly you mean by --tracing 09:58
tellable6 2021-05-04T00:07:03Z #raku <japhb> timotimo Can you merge github.com/timo/json_fast/pull/73 ?
timotimo i would probably recommend gdb-ing into the process, breakpointing everything that creates files and everything that closes file descriptors, and grabbing a backtrace for each 09:59
Xliff timotimo: That's a lot of work for a bug report. 10:02
timotimo aye, a gdb script could / should be written for this task in particular
Xliff timotimo: I have no expertise in writing such a thing. 10:03
timotimo if/when we get tracy integrated with getting stack info for perl6-level frames, that could be very good
Xliff timotimo: Could you comment on 4356, then?
timotimo i'm not sure if that's actually a thing, but i was thinking just a little piece of text you could just copypaste into the gdb console
M#4356
Xliff At the very least I can say someone has at least commented on it.
timotimo oh it's an R anyway 10:04
Xliff If you have something to paste into gdb console, I can do.
timotimo i don't, it'd be a bit of searching and dayjob got me in its clutches at the mo 10:20
10:34 evalable6 left 10:36 evalable6 joined 11:26 MasterDuke left 12:02 reportable6 left 12:04 reportable6 joined 12:38 titsuki joined 13:32 dogbert11 joined 13:35 dogbert17 left 13:36 frost-lab left
Geth rakudo/master: 5 commits pushed by (Vadim Belman)++ 14:07
Xliff timotimo: OK, thanks. If you can tell me the broad strokes, maybe I can write it? 14:11
timotimo ok, so first you'd search for anything that opens files; in your case i guess it'd be enough to find those that are actually used in the workload 14:22
those places you'd want to put a breakpoint on
you can use "commands <breakpoint number>" to put some gdb commands in that should run when the breakpoint is hit; i hope you can put more than one breakpoint number in there 14:23
Xliff Um. This is occurring in precomp though.
So I'd be doing this against the rakudo source?
timotimo the commands would be like a bunch of "up" so a "tc" comes in scope, then "call MVM_dump_backtrace(tc)"
Xliff Or MoarVM? 14:24
I'm thinking MoarVM
timotimo well, you can really only gdb C code or similar, so you'll target the source code of moarvm to find places that open stuff, but you'd be running moarvm with the arguments that reproduce your workload
Xliff Even though the problem is in the Rakudo compiler.
timotimo have you been able to freeze the execution at the point the explosion happens? 14:26
Xliff Never.
It's already changed once.
timotimo at that point you could check `ls -lsa /proc/<pid of moarvm process>/fds/` or fdinfo or so to see if these fds point at anything suspicious
because on unix "everything is a file" we have more file descriptors than actual files on disk that we open 14:27
Xliff Good suggestion!
timotimo for example, every thread can get a uv eventloop, which also grabs at least one file descriptor for polling purposes
14:29 dogbert17 joined
[Tux] Rakudo v2021.04-19-g2776d6962 (v6.d) on MoarVM 2021.04-34-g684b240e0
csv-ip5xs0.852 - 0.881
csv-ip5xs-208.763 - 9.785
csv-parser27.497 - 28.828
csv-test-xs-200.381 - 0.439
test8.045 - 8.067
test-t2.021 - 2.136
test-t --race0.905 - 0.910
test-t-2033.925 - 34.731
test-t-20 --race9.759 - 9.807
14:31
Now I'll run again with teams closed. Note it was only active in the background using this timing, no actual conversations. 14:32
14:32 dogbert12 joined 14:33 dogbert11 left 14:34 dogbert17 left 14:45 dogbert17 joined 14:48 dogbert12 left 14:52 dogbert11 joined 14:55 dogbert17 left
Xliff *sigh* 15:09
OK. Found another bug while in a docker environment
"No such method 'absolute' for invocant of type 'Slip'" 15:10
^ This while in a docker container. If I do the same command from outside the container, it doesn't show.
So I suppose the bug could be environmental.
This is not the first time I've encountered it.
tonyo [Tux]: what csv file do you run that benchmark with? 15:11
Xliff I've pushed the Docker image here: hub.docker.com/repository/docker/x...glib-suite
15:13 b2gills left, b2gills joined
[Tux] Rakudo v2021.04-19-g2776d6962 (v6.d) on MoarVM 2021.04-34-g684b240e0
csv-ip5xs0.822 - 0.849
csv-ip5xs-208.537 - 9.247
csv-parser26.315 - 28.068
csv-test-xs-200.390 - 0.390
test7.735 - 8.086
test-t2.010 - 2.017
test-t --race0.895 - 1.029
test-t-2034.209 - 35.316
test-t-20 --race10.531 - 11.140
15:14
/tmp/hello.csv
file://localhost/work/www/Talks/CSV6/speed.html 15:15
tux.nl/Talks/CSV6/speed.html
tonyo danke
[Tux] See also the link at the bottom of tux.nl/Talks/CSV6/speed5.html 15:16
tonyo ahh i mean what is the csv file those things are actually parsing. i looked in your repo, did i miss it? 15:17
[Tux] that is the file hello.csv as described on speed.html. Simple as that
in the repo it is described in README.speed 15:18
tonyo okay, so the csv file isn't in the repo 15:20
rust is quick 15:21
Xliff :s 15:37
$*KERNEL.arch is "unknown" when in a Docker image
How is $*KERNEL.arch populated? 15:39
lizmat uname something? 15:40
Xliff uname -a?
From the container: "Linux 4d94d3324b85 5.8.0-50-generic #56-Ubuntu SMP Mon Apr 12 17:18:36 UTC 2021 x86_64 GNU/Linux" 15:41
Which contains the correct arch.
lizmat looks like uname -m
tbrowder [Coke]: ref MoarVM issue #1469: I added a partial log showing the error lines
lizmat ah no, uname -p ?
Xliff Actually, I think it's getting the value from "uname -p"
lizmat yeah 15:42
Xliff lizmat: Jinx! :)
Should it be `uname -m`?
lizmat I haz no idea
Xliff What's the best way to test to see if you are running in a Docker image?
lizmat jmerelo might know
lizmat doesn't
tonyo Xliff: ls /.dockerenv 15:44
does that file exist in your image?
Xliff Yyp 15:45
tonyo ideally you'd set an env var.
Xliff Yup, even
tonyo if that file exists or if there's a docker user in proc/user/cgroup? i think that's the path 15:46
i'd set an environment variable though.
s/group/user
Xliff tonyo++: OK. That worked! Thanks 15:56
16:15 Kaiepi left 16:17 Kaiepi joined, Kaiepi left 16:18 Kaiepi joined 16:25 domidumont left 16:39 Kaiepi left
Geth rakudo: e6a7bfe7bb | (Elizabeth Mattijsen)++ | src/core.c/set_elem.pm6
Make infix (elem) about 30x as fast on native arrays

  - Str on str arrays
  - Int on int arrays
Most of the speed win is caused by:
  - not needing to do any .WHICH on anything
... (5 more lines)
17:18
17:45 patrickb joined 18:01 reportable6 left 18:04 reportable6 joined
sena_kun any ideas on what's happening in github.com/rakudo/rakudo/issues/4357 ? 18:37
lizmat added MacOS output, which matched Linux 18:40
18:40 titsuki left
sena_kun committable6, note $*TMPDIR.child("../\x[308]").resolve(:completely).absolute.raku; 18:59
committable6 sena_kun, ¦note: «Cannot find this revision (did you mean “coke”?)»
sena_kun bisectable6, note $*TMPDIR.child("../\x[308]").resolve(:completely).absolute.raku;
bisectable6 sena_kun, Will bisect the whole range automagically because no endpoints were provided, hang tight
sena_kun, Output on all releases: gist.github.com/5c6e76138eb7ac9326...75f6811965 19:00
sena_kun, Bisecting by exit code (old=2019.07.1 new=2019.11). Old exit code: 1
sena_kun, bisect log: gist.github.com/9458ad81aa72a4d64c...04f17b570e
sena_kun, (2019-11-01) github.com/rakudo/rakudo/commit/7c...3c0a81e7ad
sena_kun, Bisecting by output (old=2017.05 new=2017.06) because on both starting points the exit code is 1
sena_kun bisectable6, note $*TMPDIR.child("../\x[308]").resolve(:completely).absolute.perl; 19:01
bisectable6 sena_kun, Will bisect the whole range automagically because no endpoints were provided, hang tight
sena_kun, bisect log: gist.github.com/b2fd7a6ebd62406e56...56e14c75cb
sena_kun, (2017-06-01) github.com/rakudo/rakudo/commit/7c...656b955cd7
sena_kun, Bisecting by output (old=2016.09 new=2016.10) because on both starting points the exit code is 1
sena_kun, bisect log: gist.github.com/f1313fa496c5d689ca...dd733c00c6
sena_kun, (2016-09-27) github.com/rakudo/rakudo/commit/22...0f14b9c05c
sena_kun, Output on all releases: gist.github.com/d2437fbb922ec363a8...9fb1b1efa6
sena_kun, Bisecting by output (old=2017.03 new=2017.04.3) because on both starting points the exit code is 0
sena_kun, Output on all releases and bisected commits: gist.github.com/7e82563cd86c33a635...1b485faed5
sena_kun, bisect log: gist.github.com/c4884b8863fc185795...129a224e4f
sena_kun, (2017-04-10) github.com/rakudo/rakudo/commit/6a...82658b4662
sena_kun, ⚠ New output detected, please review the results manually 19:02
sena_kun, Output on all releases and bisected commits: gist.github.com/7380fc9e9e00d27e5f...cbc7921de3
sena_kun who wants to investigate now? :) 19:05
19:07 cog_ joined 19:10 cog left
tonyo sounds like a job for sena_kun 19:11
lizmat sena_kun: that commit is just exposing the issue
sena_kun lizmat, well, yes. 19:16
anyway, quite a lot of packages are broken on Windows due to this bug, so IMO it would be nice to fix it and unbreak lots of code at once 19:17
lizmat ah, indeed...
but maybe that's just how WIndows works with directories ?
re encoding and such ?
sena_kun lizmat, dunno, it feels like the resolving of a `..` in a path is bogus on Windows in some edge case (related to this diacritic combiner, apparently) while it works fine on gnu/linux 19:19
lizmat yeah, that feels about right :-(
19:20 MasterDuke joined 19:38 dogbert17 joined 19:42 dogbert11 left 19:46 dogbert17 left 20:08 Xliff left 20:09 dogbert17 joined, dogbert11 joined 20:13 dogbert17 left 20:19 dogbert11 left 20:20 dogbert11 joined 20:29 dogbert11 left, MasterDuke left 20:32 dogbert11 joined, patrickb left 21:24 Kaiepi joined 22:39 dogbert17 joined 22:42 dogbert11 left 23:23 Xliff joined
Xliff \o 23:23
I've updated #4356 with instructions to (hopefully) access the issue inside of a Docker image. The bug writeup contains a new set of replication steps in the comments. 23:24
Could someone please look at it when they get a chance. This bug is really killing forward project with my efforts. Thanks. 23:25
^^ timotimo nine vrurg MasterDuke ugexe lizmat
tonyo Xliff: what's your `ulimit -n` ? 23:28
Xliff 1024 23:29
That actually should be unlimited.... wtf? 23:30
tonyo woshub.com/too-many-open-files-error-linux/
you might check that ^
that docker image is huge : ) 23:31
Xliff Yes. That way all of the p6-GLib stuff is precompiled. 23:32
Better than downloading all of the repos and compiling them by hand... :/
tonyo you should change your ulimit and give that precomp a shot 23:34
Xliff tonyo++: Doing that right now. Thanks! 23:36
tonyo doesn't mean it's not leaking handles but it does help golf that down to a small repo and setting a low ulimit 23:40
[Coke] (did you mean coke?) huh? 23:41
Xliff tonyo: That's hopeful given the amount of code that I have. It might not be possible to get a small golf. 23:57
That's what always worries me. 23:58