[00:07] *** reportable6 left [00:10] *** reportable6 joined [00:35] *** kjp left [00:47] that sounds like a good idea actually, yeah [02:18] *** sourceable6 left [02:18] *** bloatable6 left [02:18] *** notable6 left [02:18] *** bisectable6 left [02:18] *** benchable6 left [02:18] *** unicodable6 left [02:18] *** greppable6 left [02:18] *** squashable6 left [02:18] *** linkable6 left [02:18] *** statisfiable6 left [02:18] *** quotable6 left [02:18] *** shareable6 left [02:18] *** tellable6 left [02:18] *** evalable6 left [02:18] *** coverable6 left [02:18] *** nativecallable6 left [02:18] *** releasable6 left [02:18] *** reportable6 left [02:18] *** committable6 left [02:19] *** bisectable6 joined [02:19] *** greppable6 joined [02:19] *** quotable6 joined [02:19] *** notable6 joined [02:20] *** unicodable6 joined [02:20] *** coverable6 joined [02:20] *** shareable6 joined [02:20] *** reportable6 joined [03:19] *** bloatable6 joined [03:19] *** evalable6 joined [03:19] *** committable6 joined [03:20] *** statisfiable6 joined [03:21] *** linkable6 joined [03:21] *** sourceable6 joined [04:08] *** kjp joined [04:20] *** tellable6 joined [04:20] *** nativecallable6 joined [04:20] *** releasable6 joined [04:20] *** benchable6 joined [05:22] *** evalable6 left [05:22] *** committable6 left [05:22] *** tellable6 left [05:22] *** nativecallable6 left [05:22] *** greppable6 left [05:22] *** benchable6 left [05:22] *** bisectable6 left [05:22] *** statisfiable6 left [05:22] *** releasable6 left [05:22] *** notable6 left [05:22] *** sourceable6 left [05:22] *** linkable6 left [05:22] *** reportable6 left [05:22] *** bloatable6 left [05:22] *** shareable6 left [05:22] *** quotable6 left [05:22] *** unicodable6 left [05:22] *** coverable6 left [05:23] *** coverable6 joined [05:23] *** unicodable6 joined [05:23] *** bisectable6 joined [05:24] *** committable6 joined [05:24] *** quotable6 joined [05:24] *** sourceable6 joined [05:25] *** benchable6 joined [05:25] *** evalable6 joined [05:25] *** linkable6 joined [05:25] *** releasable6 joined [06:24] *** tellable6 joined [06:25] *** notable6 joined [06:25] *** nativecallable6 joined [06:57] *** bartolin left [06:59] *** bartolin joined [07:21] *** squashable6 joined [08:19] ¦ MoarVM: MasterDuke17++ created pull request #1633: Simplify MVM_string_utf8_encode_C_string [08:19] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/1633 [08:23] *** statisfiable6 joined [08:24] *** greppable6 joined [08:25] *** bloatable6 joined [08:34] i hope we're not using that function in performance-critical code tho :) [08:36] it's used in a few more places than i expected, but i don't think any are terribly performance critical in "usual" code [08:40] samcv: you might find https://github.com/dotnet/runtime/pull/63285 and/or https://github.com/smart-tool/smart interesting [08:42] timo: while you're here, any thoughts about my get-and-reset-profiling-data-with-a-syscall experiment? is the naive MVM_profile_end+MVM_profile_start just not even going to work for experimentation and i really need to implement a correct solution? [08:43] i'm also stumped why the profile_end would return an empty object, though i didn't look at the actual code yet [08:43] Is realloc actually slower than malloc + memcpy? [08:44] realloc should sometimes be able to just re-use the memory and not memcpy [08:44] when the allocated piece was already big enough to hold one more byte anyway [08:48] i'd suggest that unless the realloc is in fact quite a bit slower (which seems unlikely), it's still a logical simplification of the code [08:49] nine: btw, have you seen that there have been a couple recent CI fails in t/02-rakudo/reproducible-builds.t ? [08:49] we don't implement realloc ourselves, right? only in like the fsa and such? [08:50] correct [08:59] nine: think it would make sense to add something like `MVM_dump_backtrace(tc)` to the hash iterators in moarvm, compile nqp and rakudo, and see where they're being called from without being sorted afterwards? [09:01] definitely [09:08] *** reportable6 joined [09:20] ¦ MoarVM: 99e0a715d9 | (Daniel Green)++ | src/strings/utf8.c [09:20] ¦ MoarVM: Simplify MVM_string_utf8_encode_C_string [09:20] ¦ MoarVM: [09:20] ¦ MoarVM: By just reallocing the result of MVM_string_utf8_encode and sticking a [09:20] ¦ MoarVM: NULL on the end of it instead of mallocing a new larger string and [09:20] ¦ MoarVM: memcpying into it. [09:20] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/99e0a715d9 [09:20] ¦ MoarVM: 76051fcf3c | MasterDuke17++ (committed using GitHub Web editor) | src/strings/utf8.c [09:20] ¦ MoarVM: Merge pull request #1633 from MasterDuke17/simplify_MVM_string_utf8_encode_C_string [09:20] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/76051fcf3c [09:25] *** shareable6 joined [09:29] while we're discussing PRs, any comments on https://github.com/MoarVM/MoarVM/pull/1632 ? [09:35] ¦ MoarVM: f4b4692ae4 | (Daniel Green)++ | .gitmodules [09:35] ¦ MoarVM: Ignore untracked changes in the ryu submodule [09:35] ¦ MoarVM: [09:35] ¦ MoarVM: This stops the `modified: 3rdparty/ryu (untracked content)` message [09:35] ¦ MoarVM: from showing up in a `git status` in the MoarVM repo after it's built. [09:35] ¦ MoarVM: An alternative would be to add `ryu/ds2.o` to 3rdparty/ryu/.gitignore, [09:35] ¦ MoarVM: but that would require adding a commit to the submodule instead. [09:35] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f4b4692ae4 [09:35] ¦ MoarVM: 39e9a6be0c | MasterDuke17++ (committed using GitHub Web editor) | .gitmodules [09:35] ¦ MoarVM: Merge pull request #1632 from MasterDuke17/ignore_untracked_changes_in_ryu_submodule [09:35] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/39e9a6be0c [09:51] hm. i have 470 distinct `at <...>` lines for the nqp build and 535 for rakudo (but the build was killed before stage parse finished). i'm not sure how to tell which ones to look into [09:55] e.g., could https://github.com/rakudo/rakudo/blob/master/src/Perl6/World.nqp#L710-L712 be problematic? [09:56] No, it just binds those keys in another hash [10:14] hm, this is the only fail i could quickly find in the CI logs, but maybe it's not actually a reproducibility problem? https://dev.azure.com/Rakudo/rakudo/_build/results?buildId=2114&view=logs&j=5b519533-8a6a-5ec0-9930-bcff75bff664&t=d65b4135-4d4a-54ef-1dd3-d5b6c7b4a3e7&l=2058 [10:24] 2022-01-03T22:07:44.5816545Z ===SORRY!=== Error while compiling D:\a\1\rakudo/t/02-rakudo/reproducible-builds.t [10:24] That doesn't look like it's due to an actual reproducible build issue [10:24] unrelated, but https://github.com/rakudo/rakudo/blob/master/t/04-nativecall/CompileTestLib.rakumod#L51 is using the deprecated `.status` [10:25] Why is .status deprecated in the first place? [10:25] yeah. i'm going to put this investigation on hold until/unless we see another real reproducibility fail [10:26] from the deprecation message `Please use exitcode and/or signal methods (status is to be removed in 2022.06) instead.` [10:27] *** frost left [10:27] But exitcode and signal do different things than status [10:28] https://github.com/rakudo/rakudo/pull/4442 [10:29] https://github.com/rakudo/rakudo/issues/1366 and https://github.com/Raku/doc/issues/1710 [10:29] looks like a question for moritz [10:29] None of that explains why it has to go. Only that it's not wanted. [10:32] istr something about not all platforms setting the right values, so you need to inspect the two parts manually depending on what os you're on? [10:37] *** frost joined [10:39] some relevant chat here https://colabti.org/irclogger/irclogger_log/raku-dev?date=2021-07-27#l103 [12:04] *** reportable6 left [12:04] *** linkable6 left [12:05] *** linkable6 joined [12:06] *** reportable6 joined [12:22] oh interesting. i fired up my mimalloc branch again and everything built ok. i think it's because previously i had changed the runners to use the MVM_* functions instead of e.g., malloc directly. however, that really shouldn't be required to get the benefit of using mimalloc, so i'll probably just leave that for a potential future change [12:24] i guess the thing to do now is decide if i/we only want to use mimalloc if a probe in Configure for cmake succeeds [12:25] however, i just recently noticed https://github.com/microsoft/mimalloc#single-source which may mean we wouldn't need to depend upon cmake at all [12:41] *** linkable6 left [13:14] MasterDuke: oh, that sounds way better :) I guess mimalloc would then be just another part of libmoar.so [13:16] i assume that would increase the size of libmoar.so by the same amount as libmimalloc.a (234k), but the tradeoff seems worth it [13:17] now trying to figure out how to reconfigure the build to just build that single file [13:17] I'm pretty sure that's smaller than the unicode tables :P [13:18] heh [13:20] MasterDuke: I think all it needs is -I3rdparty/mimalloc/include -I3rdparty/mimalloc/src in CFLAGS and 3rdparty/mimalloc/src/static.c in the sources list for libmoar [13:21] i was wondering about that. it looks like for some of the other single file 3rdparty libs (e.g., tinymt) we do create .a's [13:39] oh, everything seems to be fine now [13:40] a downside however...mimalloc appears to not (yet) play nicely with valgrind [13:40] *** frost left [13:41] https://github.com/microsoft/mimalloc/issues/251 [13:43] *** linkable6 joined [13:44] *** linkable6 left [13:45] *** linkable6 joined [13:46] *** Geth left [13:46] *** Geth joined [13:57] *** Geth left [13:57] *** Geth joined [14:00] hm. but now i get segfaults with --full-cleanup [14:01] *** Geth left [14:01] *** Geth joined [14:05] *** Geth left [14:05] *** Geth joined [14:10] *** Geth left [14:10] *** Geth joined [14:13] Can we disable mimalloc in valgrind builds? [14:13] well, it looks like someone is working on integrating mimalloc with valgrind's api [14:14] but yeah, i guess that could be a workaround for now [14:14] *** Geth left [14:14] *** Geth joined [14:14] but first i need to figure out why i'm getting these segfaults now [14:17] happening in MVM_gc_global_destruction [14:25] so far always seems to be here https://github.com/MoarVM/MoarVM/blob/master/src/6model/6model.c#L60 [14:26] and the st looks ok. debug_name = 0x555555559c60 "BOOTStrArray" [14:26] but there's a warning printed before the segv: `mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x555555559c60` [14:29] hm. the other fields do have addresses that look more like this though `boolification_spec = 0x200004e0760` [14:31] looks like st->debug_name can be set by MVM_serialization_read_cstr, MVM_string_utf8_encode_C_string, and strdup [14:32] wonder if it's the strdup causing the problem, since i'm not using mimalloc to override the usual functions [14:34] ah-ha! they provide a `mi_strdup` [14:41] and now spectests are good [14:42] Does it change setting compilation times? [14:47] yes. doesn't seem to speed up a spectest as much now, but that might because only libmoar is using it, not everything when i LD_PRELOADed it [14:49] there have been some changes to moarvm and rakudo since i ran the numbers in https://github.com/MoarVM/MoarVM/issues/1623, but it looks like it's still about 1-1.5s faster at stage parse [14:49] without the change to convert the fsa to just forward on to the regular functions [14:54] nine: btw, do you happen to have any benchmarking code i could try like jnthnwrthngtn mentioned in his comment on that issue? [14:54] just tried with the sort-of-disable-fsa patch, times seem pretty much the same (i.e., mimalloc w/ normal fsa is same as mimalloc w/ disabled fsa) [14:55] so with this we could even rip out the fsa completely [15:17] MasterDuke: sorry, benchmark availability has long been a sore point for Rakudo [15:19] yeah. another long-standing project i contemplate occasionally is to get that daily moarvm bench back up [15:22] ¦ MoarVM: MasterDuke17++ created pull request #1634: Add mimalloc submodule [15:22] ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/1634 [16:03] huh, `3rdparty\mimalloc\src\static.o:static.c:(.text+0x451d): undefined reference to `BCryptGenRandom'` [16:06] afk for a while, will have to see about that this evening [16:18] ok, this is weird: trying to get IO::Path!SET-SELF below inlining limit [16:18] with this signature: method !SET-SELF(str $path, IO::Spec $SPEC, Str:D $CWD) [16:18] the size is 340 bytes [16:19] with this signature: method !SET-SELF(str $path, $SPEC, $CWD) [16:19] the size is 446 bytes [16:20] how can the bytesize **increase** by removing constraints in the signature ? [16:36] That can only be answered by looking at the actual generated speshed bytecode [16:54] *** linkable6 left [16:54] *** linkable6 joined [16:57] *** Geth left [16:57] *** Geth joined [17:09] lizmat: When the type is not iterable, it emits smaller code because it assumes it will not need to defensively place it into a Scalar container [18:07] *** reportable6 left [20:10] *** reportable6 joined [20:17] jnthnwrthngtn: Oh that is very valuable information. I had been avoiding putting types on super-frequently-called methods under the assumption that there was extra cost there ... but it sounds like I may have been preventing inlining. [20:35] *** [Coke] left [20:39] *** [Coke] joined [22:13] *** sena_kun left [22:15] *** sena_kun joined [22:26] ugh, now dealing with windows build problems again. guess it's soon going to be time to break out that vm of mine instead of waiting for ci jobs to run... [22:27] especially since there still doesn't seem to be a way to specify the jobs to start first (windows and jvm, since those take the longest) [23:03] ok, appeared to have solved the mingw bcrypt problem. now just `inst-rakudo.exe.o:main.c:(.text+0x84): undefined reference to `mi_free'` on windows (mingw and msvc) to fix [23:04] <[Coke]> MasterDuke++ [23:06] it will be nice to get back to projects that don't involve the build system [23:07] <[Coke]> :) [23:10] oh, this last one may actually be a pretty trivial fix, let's see... [23:15] https://github.com/rakudo/rakudo/commit/1b5c0b9516 [23:15] now to wait a couple min to restart the CI jobs in the mimalloc PR and hope they pull in that rakudo commit [23:36] ah ha! [23:37] Nicholas: do you mind testing https://github.com/MoarVM/MoarVM/pull/1634 on some of the crazier systems you have access to?