|
Welcome to the main channel on the development of MoarVM, a virtual machine for NQP and Rakudo (moarvm.org). This channel is being logged for historical purposes. Set by lizmat on 24 May 2021. |
|||
|
00:07
reportable6 left
00:10
reportable6 joined
00:35
kjp left
|
|||
| timo | that sounds like a good idea actually, yeah | 00:47 | |
|
02:18
sourceable6 left,
bloatable6 left,
notable6 left,
bisectable6 left,
benchable6 left,
unicodable6 left,
greppable6 left,
squashable6 left,
linkable6 left,
statisfiable6 left,
quotable6 left,
shareable6 left,
tellable6 left,
evalable6 left,
coverable6 left,
nativecallable6 left,
releasable6 left,
reportable6 left,
committable6 left
02:19
bisectable6 joined,
greppable6 joined,
quotable6 joined,
notable6 joined
02:20
unicodable6 joined,
coverable6 joined,
shareable6 joined,
reportable6 joined
03:19
bloatable6 joined,
evalable6 joined,
committable6 joined
03:20
statisfiable6 joined
03:21
linkable6 joined,
sourceable6 joined
04:08
kjp joined
04:20
tellable6 joined,
nativecallable6 joined,
releasable6 joined,
benchable6 joined
05:22
evalable6 left,
committable6 left,
tellable6 left,
nativecallable6 left,
greppable6 left,
benchable6 left,
bisectable6 left,
statisfiable6 left,
releasable6 left,
notable6 left,
sourceable6 left,
linkable6 left,
reportable6 left,
bloatable6 left,
shareable6 left,
quotable6 left,
unicodable6 left,
coverable6 left
05:23
coverable6 joined,
unicodable6 joined,
bisectable6 joined
05:24
committable6 joined,
quotable6 joined,
sourceable6 joined
05:25
benchable6 joined,
evalable6 joined,
linkable6 joined,
releasable6 joined
06:24
tellable6 joined
06:25
notable6 joined,
nativecallable6 joined
06:57
bartolin left
06:59
bartolin joined
07:21
squashable6 joined
|
|||
| Geth | MoarVM: MasterDuke17++ created pull request #1633: Simplify MVM_string_utf8_encode_C_string |
08:19 | |
|
08:23
statisfiable6 joined
08:24
greppable6 joined
08:25
bloatable6 joined
|
|||
| timo | i hope we're not using that function in performance-critical code tho :) | 08:34 | |
| MasterDuke | it's used in a few more places than i expected, but i don't think any are terribly performance critical in "usual" code | 08:36 | |
| samcv: you might find github.com/dotnet/runtime/pull/63285 and/or github.com/smart-tool/smart interesting | 08:40 | ||
| timo: while you're here, any thoughts about my get-and-reset-profiling-data-with-a-syscall experiment? is the naive MVM_profile_end+MVM_profile_start just not even going to work for experimentation and i really need to implement a correct solution? | 08:42 | ||
| timo | i'm also stumped why the profile_end would return an empty object, though i didn't look at the actual code yet | 08:43 | |
| nine | Is realloc actually slower than malloc + memcpy? | ||
| timo | realloc should sometimes be able to just re-use the memory and not memcpy | 08:44 | |
| when the allocated piece was already big enough to hold one more byte anyway | |||
| MasterDuke | i'd suggest that unless the realloc is in fact quite a bit slower (which seems unlikely), it's still a logical simplification of the code | 08:48 | |
| nine: btw, have you seen that there have been a couple recent CI fails in t/02-rakudo/reproducible-builds.t ? | 08:49 | ||
| timo | we don't implement realloc ourselves, right? only in like the fsa and such? | ||
| MasterDuke | correct | 08:50 | |
| nine: think it would make sense to add something like `MVM_dump_backtrace(tc)` to the hash iterators in moarvm, compile nqp and rakudo, and see where they're being called from without being sorted afterwards? | 08:59 | ||
| nine | definitely | 09:01 | |
|
09:08
reportable6 joined
|
|||
| Geth | MoarVM: 99e0a715d9 | (Daniel Green)++ | src/strings/utf8.c Simplify MVM_string_utf8_encode_C_string By just reallocing the result of MVM_string_utf8_encode and sticking a NULL on the end of it instead of mallocing a new larger string and memcpying into it. |
09:20 | |
| MoarVM: 76051fcf3c | MasterDuke17++ (committed using GitHub Web editor) | src/strings/utf8.c Merge pull request #1633 from MasterDuke17/simplify_MVM_string_utf8_encode_C_string |
|||
|
09:25
shareable6 joined
|
|||
| MasterDuke | while we're discussing PRs, any comments on github.com/MoarVM/MoarVM/pull/1632 ? | 09:29 | |
| Geth | MoarVM: f4b4692ae4 | (Daniel Green)++ | .gitmodules Ignore untracked changes in the ryu submodule This stops the `modified: 3rdparty/ryu (untracked content)` message from showing up in a `git status` in the MoarVM repo after it's built. An alternative would be to add `ryu/ds2.o` to 3rdparty/ryu/.gitignore, but that would require adding a commit to the submodule instead. |
09:35 | |
| MoarVM: 39e9a6be0c | MasterDuke17++ (committed using GitHub Web editor) | .gitmodules Merge pull request #1632 from MasterDuke17/ignore_untracked_changes_in_ryu_submodule |
|||
| MasterDuke | hm. i have 470 distinct `at <...>` lines for the nqp build and 535 for rakudo (but the build was killed before stage parse finished). i'm not sure how to tell which ones to look into | 09:51 | |
| e.g., could github.com/rakudo/rakudo/blob/mast...#L710-L712 be problematic? | 09:55 | ||
| nine | No, it just binds those keys in another hash | 09:56 | |
| MasterDuke | hm, this is the only fail i could quickly find in the CI logs, but maybe it's not actually a reproducibility problem? dev.azure.com/Rakudo/rakudo/_build...amp;l=2058 | 10:14 | |
| nine | 2022-01-03T22:07:44.5816545Z ===SORRY!=== Error while compiling D:\a\1\rakudo/t/02-rakudo/reproducible-builds.t | 10:24 | |
| That doesn't look like it's due to an actual reproducible build issue | |||
| MasterDuke | unrelated, but github.com/rakudo/rakudo/blob/mast...akumod#L51 is using the deprecated `.status` | ||
| nine | Why is .status deprecated in the first place? | 10:25 | |
| MasterDuke | yeah. i'm going to put this investigation on hold until/unless we see another real reproducibility fail | ||
| from the deprecation message `Please use exitcode and/or signal methods (status is to be removed in 2022.06) instead.` | 10:26 | ||
|
10:27
frost left
|
|||
| nine | But exitcode and signal do different things than status | 10:27 | |
| MasterDuke | github.com/rakudo/rakudo/pull/4442 | 10:28 | |
| github.com/rakudo/rakudo/issues/1366 and github.com/Raku/doc/issues/1710 | 10:29 | ||
| looks like a question for moritz | |||
| nine | None of that explains why it has to go. Only that it's not wanted. | ||
| MasterDuke | istr something about not all platforms setting the right values, so you need to inspect the two parts manually depending on what os you're on? | 10:32 | |
|
10:37
frost joined
|
|||
| MasterDuke | some relevant chat here colabti.org/irclogger/irclogger_lo...07-27#l103 | 10:39 | |
|
12:04
reportable6 left,
linkable6 left
12:05
linkable6 joined
12:06
reportable6 joined
|
|||
| MasterDuke | oh interesting. i fired up my mimalloc branch again and everything built ok. i think it's because previously i had changed the runners to use the MVM_* functions instead of e.g., malloc directly. however, that really shouldn't be required to get the benefit of using mimalloc, so i'll probably just leave that for a potential future change | 12:22 | |
| i guess the thing to do now is decide if i/we only want to use mimalloc if a probe in Configure for cmake succeeds | 12:24 | ||
| however, i just recently noticed github.com/microsoft/mimalloc#single-source which may mean we wouldn't need to depend upon cmake at all | 12:25 | ||
|
12:41
linkable6 left
|
|||
| nine | MasterDuke: oh, that sounds way better :) I guess mimalloc would then be just another part of libmoar.so | 13:14 | |
| MasterDuke | i assume that would increase the size of libmoar.so by the same amount as libmimalloc.a (234k), but the tradeoff seems worth it | 13:16 | |
| now trying to figure out how to reconfigure the build to just build that single file | 13:17 | ||
| moon-child | I'm pretty sure that's smaller than the unicode tables :P | ||
| MasterDuke | heh | 13:18 | |
| nine | MasterDuke: I think all it needs is -I3rdparty/mimalloc/include -I3rdparty/mimalloc/src in CFLAGS and 3rdparty/mimalloc/src/static.c in the sources list for libmoar | 13:20 | |
| MasterDuke | i was wondering about that. it looks like for some of the other single file 3rdparty libs (e.g., tinymt) we do create .a's | 13:21 | |
| oh, everything seems to be fine now | 13:39 | ||
| a downside however...mimalloc appears to not (yet) play nicely with valgrind | 13:40 | ||
|
13:40
frost left
|
|||
| MasterDuke | github.com/microsoft/mimalloc/issues/251 | 13:41 | |
|
13:43
linkable6 joined
13:44
linkable6 left
13:45
linkable6 joined
13:46
Geth left,
Geth joined
13:57
Geth left,
Geth joined
|
|||
| MasterDuke | hm. but now i get segfaults with --full-cleanup | 14:00 | |
|
14:01
Geth left,
Geth joined
14:05
Geth left,
Geth joined
14:10
Geth left,
Geth joined
|
|||
| nine | Can we disable mimalloc in valgrind builds? | 14:13 | |
| MasterDuke | well, it looks like someone is working on integrating mimalloc with valgrind's api | ||
| but yeah, i guess that could be a workaround for now | 14:14 | ||
|
14:14
Geth left,
Geth joined
|
|||
| MasterDuke | but first i need to figure out why i'm getting these segfaults now | 14:14 | |
| happening in MVM_gc_global_destruction | 14:17 | ||
| so far always seems to be here github.com/MoarVM/MoarVM/blob/mast...odel.c#L60 | 14:25 | ||
| and the st looks ok. debug_name = 0x555555559c60 "BOOTStrArray" | 14:26 | ||
| but there's a warning printed before the segv: `mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x555555559c60` | |||
| hm. the other fields do have addresses that look more like this though `boolification_spec = 0x200004e0760` | 14:29 | ||
| looks like st->debug_name can be set by MVM_serialization_read_cstr, MVM_string_utf8_encode_C_string, and strdup | 14:31 | ||
| wonder if it's the strdup causing the problem, since i'm not using mimalloc to override the usual functions | 14:32 | ||
| ah-ha! they provide a `mi_strdup` | 14:34 | ||
| and now spectests are good | 14:41 | ||
| nine | Does it change setting compilation times? | 14:42 | |
| MasterDuke | yes. doesn't seem to speed up a spectest as much now, but that might because only libmoar is using it, not everything when i LD_PRELOADed it | 14:47 | |
| there have been some changes to moarvm and rakudo since i ran the numbers in github.com/MoarVM/MoarVM/issues/1623, but it looks like it's still about 1-1.5s faster at stage parse | 14:49 | ||
| without the change to convert the fsa to just forward on to the regular functions | |||
| nine: btw, do you happen to have any benchmarking code i could try like jnthnwrthngtn mentioned in his comment on that issue? | 14:54 | ||
| just tried with the sort-of-disable-fsa patch, times seem pretty much the same (i.e., mimalloc w/ normal fsa is same as mimalloc w/ disabled fsa) | |||
| so with this we could even rip out the fsa completely | 14:55 | ||
| nine | MasterDuke: sorry, benchmark availability has long been a sore point for Rakudo | 15:17 | |
| MasterDuke | yeah. another long-standing project i contemplate occasionally is to get that daily moarvm bench back up | 15:19 | |
| Geth | MoarVM: MasterDuke17++ created pull request #1634: Add mimalloc submodule |
15:22 | |
| MasterDuke | huh, `3rdparty\mimalloc\src\static.o:static.c:(.text+0x451d): undefined reference to `BCryptGenRandom'` | 16:03 | |
| afk for a while, will have to see about that this evening | 16:06 | ||
| lizmat | ok, this is weird: trying to get IO::Path!SET-SELF below inlining limit | 16:18 | |
| with this signature: method !SET-SELF(str $path, IO::Spec $SPEC, Str:D $CWD) | |||
| the size is 340 bytes | |||
| with this signature: method !SET-SELF(str $path, $SPEC, $CWD) | 16:19 | ||
| the size is 446 bytes | |||
| how can the bytesize **increase** by removing constraints in the signature ? | 16:20 | ||
| nine | That can only be answered by looking at the actual generated speshed bytecode | 16:36 | |
|
16:54
linkable6 left,
linkable6 joined
16:57
Geth left,
Geth joined
|
|||
| jnthnwrthngtn | lizmat: When the type is not iterable, it emits smaller code because it assumes it will not need to defensively place it into a Scalar container | 17:09 | |
|
18:07
reportable6 left
20:10
reportable6 joined
|
|||
| japhb | jnthnwrthngtn: Oh that is very valuable information. I had been avoiding putting types on super-frequently-called methods under the assumption that there was extra cost there ... but it sounds like I may have been preventing inlining. | 20:17 | |
|
20:35
[Coke] left
20:39
[Coke] joined
22:13
sena_kun left
22:15
sena_kun joined
|
|||
| MasterDuke | ugh, now dealing with windows build problems again. guess it's soon going to be time to break out that vm of mine instead of waiting for ci jobs to run... | 22:26 | |
| especially since there still doesn't seem to be a way to specify the jobs to start first (windows and jvm, since those take the longest) | 22:27 | ||
| ok, appeared to have solved the mingw bcrypt problem. now just `inst-rakudo.exe.o:main.c:(.text+0x84): undefined reference to `mi_free'` on windows (mingw and msvc) to fix | 23:03 | ||
| [Coke] | MasterDuke++ | 23:04 | |
| MasterDuke | it will be nice to get back to projects that don't involve the build system | 23:06 | |
| [Coke] | :) | 23:07 | |
| MasterDuke | oh, this last one may actually be a pretty trivial fix, let's see... | 23:10 | |
| github.com/rakudo/rakudo/commit/1b5c0b9516 | 23:15 | ||
| now to wait a couple min to restart the CI jobs in the mimalloc PR and hope they pull in that rakudo commit | |||
| ah ha! | 23:36 | ||
| Nicholas: do you mind testing github.com/MoarVM/MoarVM/pull/1634 on some of the crazier systems you have access to? | 23:37 | ||