🦋 Welcome to the IRC channel of the core developers of the Raku Programming Language (raku.org #rakulang). This channel is logged for the purpose of history keeping about its development | evalbot usage: 'm: say 3;' or /msg camelia m: ... | log inspection situation still under development | For MoarVM see #moarvm
Set by lizmat on 22 May 2021.
00:30 linkable6 left, unicodable6 left, tellable6 left, evalable6 left, benchable6 left, greppable6 left, evalable6 joined 00:31 unicodable6 joined 00:33 greppable6 joined, tellable6 joined 01:31 benchable6 joined 01:32 linkable6 joined 02:29 japhb left 02:36 melezhik left 02:56 squashable6 left 02:58 squashable6 joined 03:28 MasterDuke left 03:58 japhb joined 04:03 reportable6 joined
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^10_000_000 { $t = nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 10_000_000); $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long; 04:06
camelia 4.632589257
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^20_000_000 { $t = nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 10_000_000); $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long;
camelia 9.152689692
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^10_000_000 { $t = nqp::time; $long++ if $t - $prev > 10_000_000; $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long;
camelia 7.823173845
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^10_000_000 { $t = nqp::time; $long++ if $t - $prev > 5_000_000; $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long;
camelia 7.715262905
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^10_000_000 { $t = nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 5_000_000); $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long; 04:09
camelia 4.323051041
japhb OK, you can see it clearly in that last pair. I've been looking at this for a bit, and I'm mystified as to why it's happening.
I was trying to determine whether getting high-res time was stable performance-wise, so I started checking for "long" intervals (greater than N milliseconds between consecutive time reads). 04:11
I was stripping off extra layers of overlying code one piece at a time, until I was mostly just doing nqp:: ops and assignments within the loop. 04:12
That's when I noticed that turning the "long interval" check into nqp:: ops made the loop quite a bit faster, as expected ... but the number of long intervals per test went WAY up.
On my local laptop, I see that scale of difference with 10 ms long intervals; obviously whatever system camelia works on is faster, but accounting for the speed difference by dropping down to 5 ms as the "long" threshold shows it again. 04:14
Normally an nqp::time read is less than a microsecond. But sometimes it appears to be thousands of times slower. At first I assumed preemption problems, so I quiesced my system. And then checked against camelia. 04:15
The effect is still there -- the faster loop body appears to have much spikier latency.
WTH is going on here?
(FWIW, you can see the same effect with 'now' instead of 'nqp::time' as long as you fix the variable types and account for the factor of 1_000_000_000 conversion.) 04:16
moon-child I wonder if it has to do with vdso latency 04:17
are you running linux?
japhb Yes. 'vdso'? 04:18
Oh this? en.wikipedia.org/wiki/VDSO 04:19
moon-child hmm, on further thought, that doesn't make much sense. The kernel maps a page into every process containing things like the current time, so to get the time, you can just read it from that page without having ot do a ctx switch. So maybe, if you're faster, you're more likely to read a stale value from there
but that doesn't seem likely, the more I think about it
04:22 mattil joined
japhb That's kindof a cool hack -- does it map the *same* page into every process? Meaning that there is just 1 of these special pages *physically*, it's just memory mapped separately into every process? 04:23
moon-child I think so. Don't know too much about it
japhb Huh. I wonder what happens when you enable large pages 04:24
I suppose since all the copies are probably virtual, it doesn't really matter much.
05:10 mattil left 06:02 reportable6 left 06:41 MasterDuke joined
MasterDuke japhb: do you get the same results if you disable spesh? or might it be gc pauses? 06:44
nine GC pauses may actually be the explanation. 06:45
MasterDuke nqp: my int $i := 0; my int $t0 := my int $prev := nqp::time; my int $long := 0; my $t; while $i++ < 10_000_000 { $t := nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 5_000_000); $prev := $t; }; say(nqp::div_n((nqp::time() - $t0), 1_000_000_000)); say($long); 06:46
camelia 0.510087937
MasterDuke nqp: my int $i := 0; my int $t0 := my int $prev := nqp::time; my int $long := 0; my $t; while $i++ < 10_000_000 { $t := nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 1_000_000); $prev := $t; }; say(nqp::div_n((nqp::time() - $t0), 1_000_000_000)); say($long);
camelia 0.536608651
releasable6 Next release in ≈2 days and ≈11 hours. 1 blocker. Please log your changes in the ChangeLog: github.com/rakudo/rakudo/wiki/ChangeLog-Draft 07:00
08:04 reportable6 joined 08:32 AlexDaniel left, rba[m] left
bartolin btw, I'd like to get github.com/Raku/nqp/pull/734 into the next release. (It's only for the JVM backend.) I'm inclined to merge it myself, unless someone has objections. 08:34
08:35 AlexDaniel joined
MasterDuke i don't know the jvm backend well enough to review it, but no objection here 08:36
08:37 rba[m] joined
Geth nqp: 7e33586479 | (Christian Bartolomäus)++ | src/vm/jvm/QAST/Compiler.nqp
[JVM] Coerce to SixModelObject for isconcrete test

If we end up with a Java string|long|double on the stack, invoking
  'isconcrete' will blow up badly.
This doesn't help with the functionality of 'with' yet, but it seems clearer to make it a separate commit.
nqp: ad700a25f9 | (Christian Bartolomäus)++ | 3 files
[JVM] Let with/without use 'defined'

  ... instead of 'isconcrete'.
I'm not sure if my usage of InvokeDynamic is the right way to call the found 'method defined', but it seems to improve the situation a lot. ... (8 more lines)
nqp: 71899a940a | (Elizabeth Mattijsen)++ (committed using GitHub Web editor) | 3 files
Merge pull request #734 from usev6/jvm_with

  [JVM] Let with/without use 'defined'
bartolin oh, thanks ;) 09:45
lizmat bartolin++ # not giving up on JVM 09:48
MasterDuke very much so 09:49
09:56 evalable6 left, linkable6 left 11:15 frost joined 11:58 linkable6 joined 11:59 evalable6 joined 12:02 reportable6 left 12:04 reportable6 joined 12:40 frost left 14:06 evalable6 left, linkable6 left 14:08 linkable6 joined
japhb nine, MasterDuke: GC pausing for ... what? It's the nqp::ops version that has more long pauses, the variables are all native types, and the variables are declared outside the loop, so I don't think there are any objects being created? 14:09
nine japhb: but $t is not natively typed? 14:14
MasterDuke and i suspect that even if your code isn't explicitly creating object, rakudo itself may be 14:16
japhb nine: Gah, I didn't see that while copying over from my local tests (it wasn't native typed at the beginning because it was originally assigned with `now`)
m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my int $t; for ^10_000_000 { $t = nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 5_000_000); $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long; 14:17
camelia 0.544126166
japhb m: use nqp; my int $t0 = my int $prev = nqp::time; my int $long = 0; my $t; for ^10_000_000 { $t = nqp::time; $long++ if nqp::isgt_i(nqp::sub_i($t, $prev), 5_000_000); $prev = $t; }; say (nqp::time() - $t0)/1_000_000_000; say $long;
camelia 4.717587611
japhb WTH?
Wait ... creating Int objects and destroying them? 14:18
Because nqp::time > "small" bigint?
Well damn. Now I feel like an idiot. nine++ # Thank you for the second pair of eyes! 14:19
That is a rather amazing performance difference too, by fixing just that one declaration. 14:21
MasterDuke: Was there something *else* you had in mind that might be creating objects? 14:22
MasterDuke re "small" bigints: `src/6model/reprs/P6bigint.h:5:#define MVM_IS_32BIT_INT(i) ((long long)(i) >= -2147483648LL && (long long)(i) <= 2147483647LL)`
scheduler maybe?
japhb MasterDuke: Oh hmmm, that's an interesting point. I think lizmat++ did some work to reduce the object churn in there, but I dunno if she was able to eliminate it. 14:23
MasterDuke yeah, i wouldn't expect much, but i'd be mildly surprised if it was completely nothing 14:24
lizmat but I don't think the scheduler actually runs unless you're doing a start { } somewhere
if you run your code with -Msnapper it should tell you whether the scheduler actually ran 14:25
japhb lizmat: Would you be able to add GC runs to Telemetry? I probably would have noticed something off if I'd seen the GC runs ....
lizmat: I did run it that way yesterday, and I just saw a thing about the supervisor thread running the whole time 14:26
lizmat there's no interface to get that data, afaik
japhb MoarVM folk: Pretty please? ^^
lizmat hmmm "my int $i; 1 until ++$i == 100000" tells me the supervisor thread ran the whole time 14:27
that's... intriguing 14:28
that shouldn't happen afaik 14:29
ugexe japhb: just run --profile ? 14:30
lizmat profile says no allocations basically, just from startup: 0 garbage collects 14:31
104 allocations to be precise
hmmm... maybe the message from Telemetry is a false positive 14:32
nine lizmat: could it be that Telemetry itself is causing the supervisor thread to get started?
lizmat no, I made sure of that (originally 14:33
) :-)
ok, confirmed that we appear to *always* have a supervisor thread :-( 14:44
ugexe that seems logical to me 14:45
lizmat why? 14:46
nine We really shouldn't. The supervisor thread is started when someone interacts with the thread pool, i.e. queueing something or taking a job from a queue 14:47
japhb ugexe: Because my real scripts have multiple phases with different performance envelopes, and I want to see which parts are causing the GCs.
ugexe i guess i was assuming something in the core would be spawning a thread somewhere 14:48
lizmat ah, but it *is*
nine Precompilation of a module would be one such thing, through Proc::Async
lizmat for spesh and so on
nine The spesh thread is not part of the thread pool
lizmat but that is *not* the supervisor thread
e.g. the snapper module also creates a thread, *outside* of the thread pool 14:49
the supervisor does *not* get started with method !maybe-start-supervisor(--> Nil) { 14:52
so I guess my next guess is going to be that Telemetry starts the supervisor somehow 14:53
argh... found it 14:54
japhb: are you running on MacOS 14:55
looks like the initialization of $*DISTRO is calling shell() 14:56
as a Proc, but that is a Proc::Async 14:57
Geth rakudo: MasterDuke17++ created pull request #4496:
Make concurrent use of %!conc_table safe
lizmat and $*DISTRO is used by Telemetry,
14:58 squashable6 left
lizmat MasterDuke: I assume the binding is atomic ? 14:59
hmmm.. Telemetry doesn't use $*DISTRO ? 15:00
MasterDuke lizmat: that's my understanding
lizmat aaah... loading a module causes the supervisor to be started
ok, that can be fixed :-) 15:05
15:15 discord-raku-bot left 15:17 discord-raku-bot joined
Geth rakudo: 89df7f4ba9 | (Elizabeth Mattijsen)++ | 2 files
Make checking for Windows cheaper

The R:I.IS-WIN method doesn't need to actually initialized $*DISTRO, and as such will not actually shell out on some OS's, which will in turn not cause the threadpool supervisor thread to be started.
lizmat sadly, that does not appear to fix the issue, but is a good thing anyway
afk for an hour or so& 15:20
MasterDuke nine: any idea why github.com/rakudo/rakudo/pull/4496 would have all those t/02-rakudo/reproducible-builds.t failures? 15:25
japhb lizmat: Linux Mint on x64 and Raspberry Pi OS 32-bit on RPi4 15:27
ugexe raku -e "say $*KERNEL.version" 15:30
so $*KERNEL.version could potentially be a better windows detector 15:31
dunno what it says for e.g. cygwin
nine MasterDuke: I bet on nqp::scwbdisable();/enable 15:37
I think you want those around the %!conc_table := %conc_table_clone; 15:38
MasterDuke should i just put them around the whole method body?
nine The point of those are to prevent a serialization dependency on the object doing the Perl6::Metamodel::Concretization role
Too much of those is just as bad 15:39
MasterDuke k
oops 15:41
17:01 squashable6 joined
lizmat looks like signal().tap in the end starts the supervisor thread 17:05
[Tux] Rakudo v2021.07-26-g289a62f4a (v6.d) on MoarVM 2021.07-18-g343d0f37f
csv-ip5xs0.876 - 0.878
csv-ip5xs-208.844 - 9.163
csv-parser27.941 - 28.054
csv-test-xs-200.367 - 0.382
test8.410 - 8.550
test-t1.993 - 2.034
test-t --race0.981 - 1.038
test-t-2033.604 - 34.722
test-t-20 --race10.605 - 10.664
Next one will be in 2½weeks
lizmat [Tux]: have a nice holiday!
japhb [Tux]++ # Enjoy! 17:11
japhb wonders if it's possible to get new-disp merged by the time he can run another test 17:12
17:12 b2gills left
japhb lizmat: "signal().tap in the end"? The one at github.com/rakudo/rakudo/blob/mast...kumod#L939 ? 17:14
lizmat yup, and snapper.pm6 calls that sub 17:21
so maybe we need to copy snapper.rakumod to safe-snapper.rakumod, and remove the call to control-c from snapper 17:22
so you'd have the option with -Msnapper and -Msafe-snapper
but yeah, running snapper.pm6 will always start the supervisor thread in its current state 17:23
17:34 mattil joined 17:47 b2gills joined 18:01 mattil left 18:02 mattil joined, reportable6 left, reportable6 joined 18:33 nine left, camelia left 18:37 nine joined 18:39 camelia joined 18:54 kjp_ left 19:00 Altai-man left, jdv left, timo left, jdv joined, timo joined, Altai-man joined 19:08 evalable6 joined
Geth rakudo: 58ae9394ad | (Daniel Green)++ | src/Perl6/Metamodel/Concretization.nqp
Make concurrent use of %!conc_table safe

Because it's used by MRO method dispatch, doing things such as parsing a grammar in a start block can cause concurrent access and modification. Locking didn't seem to work, but clone->modify-clone->replace-with-clone does. Fixes the various hash-related MVM_oops in S17-promise/start.t.
  dogbert17++, nine++, vrurg++
rakudo: 6d529046e5 | MasterDuke17++ (committed using GitHub Web editor) | src/Perl6/Metamodel/Concretization.nqp
Merge pull request #4496 from MasterDuke17/make_concurrent_access_to_concretization_table_safe

Make concurrent use of %!conc_table safe
19:53 kjp joined 19:54 mattil left
gfldex docs.raku.org is down for me 21:07
Altai-man .tell rba docs.raku.org is not available? 21:47
tellable6 Altai-man, I'll pass your message to rba
rba . 21:54
Cloudflares maintenance has already led to outages recentrly. www.cloudflarestatus.com 21:56
For the
moment I have turned off the proxifying for docs.raku.org and raku.org. 21:57
Give DNS a few minutes time to resolvee the new ip.
Altai-man rba, thank you very much for reacting so fast! 22:08
22:23 evalable6 left, linkable6 left 22:25 evalable6 joined
pie_flavor it's still busted 23:25
23:25 linkable6 joined
rba Still most of the sites are down. Sorry. stats.raku.org 23:42
japhb rba: Still believed to be a Cloudflare sideswipe? 23:44
rba japhb: no, think there is an issue with the webhosting server atm 23:47
japhb: notified them and hope it will get fixed soon...
japhb rba++ 23:49