00:06 pyrimidine joined 01:24 pyrimidine joined 01:36 pyrimidine joined
Geth MoarVM: jeffythedragonslayer++ created pull request #532:
VMArray2		 02:33
MasterDuke huh, i tried changing Parameter's $!flags to an int32, but the heap profile shows the exact same total bytes 02:40
samcv MasterDuke, still storing it in a 64bit int i guess 02:41
that sort of makes sense
well. sometimes. not really if you store a huge number of them. then you may be unhappy
MasterDuke well, `perl6 -e 'my int @a[10_000_000]'` shows a higher max resident than `perl6 -e 'my int32 @a[10_000_000]'` 02:43
so i assume it's actually using something smaller for int32 02:46
samcv constant @array = 'foo', 'bar'; hopefully the compilier will replace @array[0] with 'foo' in the code then? 02:49
hm 02:50
i'm happy with how much faster i've gotten UCD-gen.p6 02:52
down to 5 minutes i think 02:53
MasterDuke do you have it online somewhere? 02:55
samcv yeah 02:57
github.com/samcv/UCD
give me a few min and i'll commit the last of my optimizations
i would run it with --nonames so it doesn't make the base40 names
since that will take like 5 minutes to do currently since it's not very optimized 02:58
err it's 5 minutes if you don't generate the names i mean 02:59
MasterDuke btw, is github now using your highlighting? 03:00
samcv yep
been that way for a few weeks
still need to sort out some bugs with the code that makes the point index though 03:01
MasterDuke `my $long-pvalue`, it's highlighting `long` 03:02
samcv well generates the point index if else's
hmm i don't see that on my end
on which file
ok it takes 217.0865949 seconds without making the names now :) 03:03
\o/
hopefully didn't break anything
MasterDuke UCD-gen.p6 03:05
samcv link 03:06
MasterDuke github.com/samcv/UCD/blob/master/U...en.p6#L139 03:08
samcv yeah it's fine in Atom for me. that's weird 03:09
everything is highlighted weird tbh 03:10
MasterDuke, ok just pushed latest verison 03:20
03:30 pyrimidine joined
MasterDuke whoops. `MoarVM panic: Internal error: invalid thread ID 1256009152 in GC work pass` 03:32
03:32 agentzh joined 03:36 pyrimidi_ joined
samcv uh oh 03:45
on my program?
MasterDuke yeah, while profiling. works fine without 03:50
hm, may still have a stray change in my build, let me clear all that out 03:52
huh, worked now 04:04
samcv need to make the base 40 name thing go faster 04:39
you can do --namesonly if you want only building the names btw 04:40
err --names-only
argh i broke something. oh well it was already kinda broken before but. less broken 04:46
MasterDuke, oh looks like i used .end instead of tail. hah. no wonder it screwed it up 05:03
MasterDuke close, but no cigar
and i'm outta here, later... 05:04
samcv byby
05:35 pyrimidine joined 05:52 Guest34565 joined 07:10 pyrimidine joined 07:35 pyrimidine joined 08:30 agentzh joined 08:36 pyrimidine joined 09:37 pyrimidine joined 10:33 agentzh joined 11:09 patrickz joined
timotimo MasterDuke: p6opaque is still subject to the same aligning rules as C structs, so maybe you put the int32 in front of a pointer and that made it put padding in between? 11:24
11:42 FROGGS joined
samcv timotimo, what do you think about unicmp_s having two options for the collation_level on negative one positive 11:49
so you could sort normally except have capitals first or something 11:50
instead of after lowercase
unicmp_s(a, b, neg_collation, collation, country, language)
well language comes before country but
timotimo sounds fine to me, but changing that with bootstrap and everything is a bit annoying :) 11:51
samcv for sure
timotimo unicmp hasn't been out in a user-facing release yet, right?
samcv nope
timotimo mhm 11:52
samcv unless you have some way to do it with one integer that's not insane
timotimo well, there's an idea
i mean here: 11:53
samcv could have 1 and 2 be primary and 4 and 8 be secondary, and 16 and 32 be tertiary
timotimo we can turn the op binding for unicmp_s into a smarter thing in the QASTOperationsMAST
samcv but that makes it less easy
smarter?
timotimo that introspects whether it has 5 or 6 arguments and inserts a dummy 0 in there
yeah, you can put a piece of code in, too
samcv hm
timotimo many examples in there 11:54
samcv good thing i introduced the op and gave me more time to play with it and use it
timotimo :)
samcv and rereading unicode page, it talks a lot about it being configurable
timotimo anyway, that way would allow us to only use the one-way unicmp inside nqp and negative-able unicmp in rakudo 11:55
samcv why not use negative-able in both
i mean no reason not to
timotimo have to build another stage0
samcv yeah 11:56
timotimo i claim that stage0s are the main contributor to nqp's repo size
samcv i mean we can do QAST too
yeah i will look at that i guess sometime when i have time
and also gave up on returning differnt values based on where it broke the tie
and making sure we have a 4th level configurable as well 11:57
i gotta go to bed. night! 12:04
timotimo gnite! 12:05
samcv oh and i also need to set these modes too unicode.org/reports/tr10/#Compariso...able_Table
so more to think about lol 12:06
timotimo we'll basically do all the things forever
samcv lol
those extra things make it all more complicated lol. so that will be the last thing i do, but making up a way to set it should be good
i will dream about api implementation 12:07
timotimo that sounds pretty terrible, actually %)
12:37 agentzh joined
timotimo added stats about each thread's gen2 to the vmhealth readout 12:39
it's a whole lotta text now
dogbert17 timotimo: cool, have you noticed anything interesting/suspicious with you new branch? 12:44
dogbert17 notices that running htmlify.p6 with --parallel almost works now, almost got to the end this time before bugging out 12:45
timotimo nothing interesting so far 12:46
but i also never tried it on anything serious
my test program is a vidya game :)
dogbert17 cool
timotimo jnthn: it doesn't seem like you got everything when you pulled perl6-bench; the split_string_regex microbenchmark still has the bug that i fixed on the 8th 12:50
wednesday, that is 12:51
nine ~ 12:55
yoleaux2 10 Feb 2017 05:51Z <llfourn> nine: I'm not 100%. But I've been looking at some of your commits from around lexical use time. It looks like you had to work around something similar with 'use': github.com/rakudo/rakudo/commit/85...af607897c. That sounds to me like exactly what is happening with require atm, except in INDIRECT_NAME_LOOKUP in operators.pm rahter than .find_symbol in World.
nine Any hints for debugging "SC not yet resolved; lookup failed"? As I'm working on precomp-singleprocess-resurrection, I'm quite sure it's an object from the main program escaping into the to be precompiled comp unit. So is there wa way to catch that?
timotimo i'd have to look at the moar sourcecode 12:59
can you figure out which of the ~four ones it is that explodes in your face?
it's five instances
nine MVM_sc_get_sc_object
timotimo sounds like you're not loading SCs in the right order? 13:00
nine But how could that be if the only thing the branch changed is running the perl 6 compiler in-process using evalfiles? 13:01
jnthn I think if you can breakpoint it under the debugger you might be able to figure out the handle/description of the SC it's trying to resolve, which may provide a clue 13:02
nine That's...odd dep is 0 and it blows up in the comp unit's dependencies+deserialize. One would think that at that point the first SC would be readily available? 13:06
timotimo it might also be interesting to get a moar bytecode backtrace 13:07
"call MVM_dump_backtrace(tc)"
13:08 pyrimidi_ joined
jnthn iirc, dependencies+deserialize very early on creates the SC 13:08
In the code it produces
At least, for the precomp case
timotimo maybe we can see in what SCs we're already deserializing or something? 13:09
jnthn I wonder if there's some unfortunate assumptions going on here
timotimo like, do we see a piece of the dependency graph projected onto our call stack?
jnthn I think when we know we're not precomping, then we don't spit out any SC setup/deserialize code because we already have the SC in memory 13:10
nine There are a lot of calls to MVM_sc_get_sc
timotimo try to breakpoin tonly the panic
er, the throw_adhoc
nine Did so before. Then I thought it would be interesting to be able to look closer at MVM_sc_get_sc itself. But that's just wasting time :) 13:11
timotimo ah
nine MoarVM's backtrace at MVM_exception_throw_adhoc is even a bit shorter than the one from --ll-exception 13:12
timotimo hmm 13:13
MasterDuke in src/Perl6/Metamodel/BOOTSTRAP.nqp, how would i scalary_attr() a str array? e.g., to turn `List @named_names` into `str @named_names`? 13:17
*heh. scalar_attr() 13:18
nine I clearly see cu->body.scs_to_resolve[0]->sc being NULL there
From MoarVM's perspective there shouldn't be any difference between a calling perl6 --target=mbc and a Perl 6 program running the compiler with :target<mbc> when generating the bytecode file, shouldn't there? 13:33
Now look at this! This is a moar dump of a minimal module on nom: gist.github.com/niner/0e32037010b7...4fd803162c and a dump of the same module on precomp-singleprocess: gist.github.com/niner/6c10d50c641a...b5cd99dff5 14:01
They should be virtually identical except for maybe the cuids. But they clearly aren't. There are lot's of differences including loads of bytecode diffs in dependencies+deserialize 14:02
MasterDuke heh. `SC_2 : AC32ABF7F7462E14EE588B2D0A6062A1B512D6D1` missing from the second. seems like that might be important 14:03
nine And in the branch, Frame_4 is never called independencies+deserialize and Frame_5 is completely missing. 14:05
Ooooh...maybe it thinks it's doing nested compilation. Which is sort of is to be frank. QASTCompilerMAST would in that case for example not write the call to the repo_conflict_resolver which is suspicioulsy absent from the second mbc file 14:10
14:12 colomon joined
nine Except that it doesn't 14:23
Oh, it's the other part of the condition. The compiler just doesn't know that it's in precompilation mode! 14:29
Aah, --target only turns on precomp mode if it's actually passed on a command line evalfiles won't do it for us. 14:33
And with that, the precomp files have become virtually identical :) On to the next issue... 14:41
IOninja This one's good to merge now methinks: github.com/MoarVM/MoarVM/pull/528 14:54
works for me
Geth MoarVM: nanis++ created pull request #534:
Issue #533: MoarVM does not build on `cygwin`.		 14:55
MoarVM: 8c2c627ca5 | (A. Sinan Unur)++ | Configure.pl
Issue #533: MoarVM does not build on `cygwin`.

Remove mention of `cygwin` support from `Configure.pl`.		 15:06
MoarVM: 7ff91dbbed | (Jonathan Worthington)++ | Configure.pl
Merge pull request #534 from nanis/nanis-nocygwin

Issue #533: MoarVM does not build on `cygwin`.
jnthn IOninja: Needs one small tweak; commented on it. But otherwise, yes, looks like we can merge soon :) 15:16
MasterDuke jnthn, IOninja: any idea about my BOOTSTRAP.nqp and scalar_attr() question? i've tried a couple things, but haven't figured it out yet 15:18
IOninja doesn't know 15:19
jnthn Is that List even correct? 15:20
I don't think we store those as a high-level List
I suspect that being scalar_attr is bogus
Or if we do it should just be typed Mu since it holds a low-level thing 15:21
MasterDuke i was a little surprised it worked when i tried that (making them List) a while ago 15:23
jnthn Well, it doens't mean anything when we just bind what we want there anyway :) 15:25
MasterDuke yeah. but it does complain `P6opaque: representation mismatch when storing value (of type NQPArray) to attribute (of type str)` if i make it `str`. not surprising in that case 15:28
jnthn No, 'cus an an array of str is not a str :) 15:30
MasterDuke which is reasonable. but is there a way to say it's supposed to be an array of str? 15:32
jnthn Only looking at what we bind there 15:33
github.com/rakudo/rakudo/blob/nom/....nqp#L1888 15:34
But what is that... 15:35
github.com/rakudo/rakudo/blob/nom/....nqp#L4996
Seems a boxed Str array
Not a native str array
So yeah, there's a change to save some boxes there :) 15:36
*chance
MasterDuke i also forgot that nqp doesn't support typed arrays 15:38
jnthn Yes and no
You can do it 15:39
With NQP ops
nqp::list_s() to create a str array
And then nqp::push_s, nqp::atpos_s, nqp::bindpos_s to access it
MasterDuke ah, right
timotimo did we use to be better at while_empty than perl5? 15:44
hm, maybe it really was only the native case 15:45
i thought our int2str_concat was only shaped downwards like that a long time ago, but it was improved when we got ropes? but now it's getting slower and slower the more work we pile on it
nine Serialization Error: reference to context outside of SC 15:54
Now this ^^^ sounds very much like a leaky roof of the precompiler
Geth MoarVM/vmhealth: 60377d1d17 | (Timo Paulssen)++ | 3 files
also report page counts and free items in thread's gen2
 15:58
timotimo wowza, i have a playable framerate in my shooter game when run with valgrind! 16:00
(though the vmhealth reports make very noticable dents in performance each time)
imgur.com/a/FJiDY 16:03
quite bumpy
*fascinating*, when i move my mouse, it makes it just slow enough to come to a complete standstill as long as the cursor's moving 16:05
16:07 lucasb joined
nine The above "reference to context outside of SC" happens in serialize_closure which makes me believe that it's about my $*ADD-DEPENDENCY = -> $dependency { @dependencies.push: $dependency }; 16:07
Now how can I have the precompiler communicate dependencies to the caller without leaving any references to objects outside the precompiler in there?
lucasb good saturday to everybody 16:09
about the representation renaming pull request, I made a list of the representation names vs. filenames vs. id names 16:10
gist.github.com/anonymous/cb09e405...c6512dae6b
from there, you can spot that the different ones are Null/MVMNull and AsyncTask/MVMAsyncTask 16:11
nine What do you people think about me calling the compilation steps directly, so I can ask the compiler about those dependencies before going into the mbc stage?
timotimo write it to a file :D 16:12
nine timotimo: or or or maybe to STDOUT? 16:13
timotimo that'd be a possibility
STDOUT is a file, too :P
nine I guess it could run into deadlocks with a full buffer, but that'd probably only hit jnthn ;) 16:14
.oO( successfully circlified )
timotimo knowing his luck, yeah
fwiw, our std file handles are being set to async by libuv anyway
so it's more probable that data just gets lost :D 16:15
nine For those who weren't tuned in at the time: jnthn running into the buffer issue prompted me to take another look at in-process precompilation in the first place ;)
timotimo i think we have an op that clears the sc from an object
if you pass native strings, it could be possible to detach it properly
though it could be that that clearing op properly clears whole object graphs 16:16
though of course it mustn't go through to, like, Int or something
*shrug*
*sigh*, the pushme benchmark does something different in rakudo vs nqp and perl5 16:17
nine I'll give the "ask, don't tell" thing a try. Would be nice to be able to use more than just basic strings for communication
timotimo froggs used to use that op to put whole json-like structures into files, iirc
so maybe you can do stuff like that
lucasb also, the other inconsistent name is MVM_REPR_ID_MVMNativeCall 16:19
jnthn, IOninja: so, my questions are:
should the filenames and ids of representations VMArray, VMException, VMHash, VMIter, VMThread be renamed to remove MVM?
should the repr ids of the CThings be renamed to remove the MVM from the name? (eg. MVM_REPR_ID_MVMCArray -> MVM_REPR_ID_CArray)
should those inconsistent names be renamed? (NativeCall,Null,AsyncTask)
IOninja doesn't know 16:20
lucasb IOninja: ok :) I just wanted to keep you on the loop
timotimo oh, that benchmark is fixed in a newer version than jnthn is running on his machine
lucasb IOninja: BTW, thanks for fixing the empty itemized list thing 16:23
nine Ooh... nqp::getcomp("perl6") will always return the same compiler object. So using that to compile a different module can quite possibly screw up the current compilation. 16:24
timotimo oh
right, we put some stuff in the one instance we have 16:25
like the commandline arguments, right?
though i'm not sure the compiler actually holds any state while it's compiling?
i mean, apart from stuff on the stack via dynvars and such
nine I don't know. And it makes me nervous. There's e.g. $!save_ctx 16:29
timotimo ooh
jnthn: i think you'll have to regenerate the perl5 results for perl6-bench to become happy, since now you're technically running the new perl6 and nqp benchmarks vs the old perl5 benchmarks, even though only the perl6 portions have changed, and i'm not sure how that changes what the graphs look like 16:30
Geth MoarVM/master: 12 commits pushed by (A. Sinan Unur)++, (Jonathan Worthington)++
review: github.com/MoarVM/MoarVM/compare/7...685af60a7b		 16:31
jnthn Merged #528; nanis++ 16:33
timotimo fantastic
jnthn timotimo: Hm, will take a look at that a little later on :)
timotimo that's why it says "perl6-bench version 0029b74, ea2a4d0" at the top of the results 16:34
16:35 pyrimidine joined 16:39 agentzh joined
TimToady jnthn: so if I get an error like "Missing serialize REPR function for REPR MVMContext (BOOTContext)", is that something that means there's just a missing function, or does it mean I've let some value through to the serializer that should never get there? 16:52
timotimo there are things that we never want to have hit the serialization stage 17:01
i forgot what Context is all about, though
TimToady it's about lexical scopes 17:08
which you'd think is something you want to remember... 17:09
but maybe the problem is that it's a BOOTContext?
timotimo is it about closures? because i thought lexical scopes was done with "outer" pointers in static frames
i assume searching irclog for "missing serialize" + MVMContext might find you something 17:10
TimToady well, I'll figure it out eventually, but can't really hack today, so I just thought I'd ask to see if it set off any sirens :) 17:11
timotimo that's fair
TimToady anyway, I think contexts are what you use to navigate all the various PseudoStashes 17:12
and since some grammars are actual objects that could have attributes, I'm trying to track the current language as an object, not just as a type 17:14
timotimo ah
TimToady but I suspect something related to that is feeding a $!ctx to the serializer somewhere
jnthn Sorry, was busy in the kitchen :) 17:26
MVMContext shows up when we do PseudoStashes or the equivalent 17:27
timotimo good to know
is there already a comment for that btw? :)
jnthn Comment where? :)
I suspect we probably can make them serializable
But it's at least a warning sign of a change unexpectedly makes us need to do so. 17:28
MasterDuke jnthn: i've been making the stuff here github.com/rakudo/rakudo/blob/nom/....nqp#L4996 an nqp::list_s(), but don't see any difference in the Parameter total bytes after a heap profile 17:39
so would it be correct to assume i'm doing something wrong i.e., if i was doing it correctly the total bytes would definitely be less? 17:40
jnthn MasterDuke: It won't make a difference to Parameter, itself, because Parameter points to the array, which is a separate object 17:41
MasterDuke: If you look at summary, however, that should show a decrease
And you can use count type="P6Str" or so 17:42
And should see less boxes
MasterDuke ahh, thanks
BOOTStr? 17:47
jnthn Oh, or that, yes :) 17:54
d'oh
Sorry, P6str is the REPR name
count repr="P6str"
But count type="BOOTStr"
:) 17:55
MasterDuke heh, figured it had to be one of the things in the top 15 17:56
18:03 lizmat joined 18:05 geekosaur joined 19:16 lizmat joined
samcv good * everyone 19:59
20:43 agentzh joined
Geth MoarVM: samcv++ created pull request #535:
Use utf8 for unicode_db files; Add missing #ifdef for variable initialization		 20:45
20:57 agentzh joined 22:44 pyrimidi_ joined
Geth MoarVM: e197d97077 | (Samantha McVey)++ | src/core/fixedsizealloc.c
Add missing #ifdef for variable initialization

This variable is unused except when #ifdef is true, so have an #ifdef for the initialization too.		 22:45
MoarVM: 26a3c52695 | (Samantha McVey)++ | 3 files
Use utf8 for unicode_db files

Fixes a compiler warning from having non-UTF-8 encoded text in unicode_db.c Also ensures when we uppercase the Emoji sequence names, letters with diacritics get uppercased properly.
MoarVM: 7a7de2f51a | (Jonathan Worthington)++ | 4 files
Merge pull request #535 from samcv/fixes

Use utf8 for unicode_db files; Add missing #ifdef for variable initialization
MasterDuke is there a way to tell the type of an nqp::list? i've converted Parameter's @!named_names to an nqp::list_s, but now this github.com/perl6/nqp/blob/master/s....nqp#L1123 is complaining 22:49
Geth MoarVM/master: 4 commits pushed by (Jeff Linahan)++, (Jonathan Worthington)++
MasterDuke if i make those two array subscripts atpos_s, it fails test #60 in t/qast/01-qast.t ('declaring and using a param with multiple names works') 22:50
jnthn ah
MasterDuke it works with just an nqp::list(), but then there's no reduction in BOOTStrs 22:51
jnthn I think we only probably only want this change for the thing we store in serialization, not in the QAST tree
Yeah, though this case only shows up in a couple of cases
*places
I'd look in lower_signature in Rakudo's Actions.nqp 22:52
MasterDuke cool, thanks
jnthn In the case it spits out a fast-path for when a named parameter has exactly two names
And just copy them into a normal array
That happens really very few times
And it's not going to affect the serialization blob
MasterDuke so just make it a list_s here? github.com/rakudo/rakudo/blob/nom/....nqp#L8121 22:54
or you mean in the else that currently just returns 0 22:56
jnthn I presume you already had to change 8119 just above it? 22:57
Anyway, no, I mean, something my my @copy; @copy[0] := nqp::atpos_s(@names, 0); @copy[1] := nqp::atpos_s(@names, 1); $var.named(@copy); or so
MasterDuke ah 22:58
22:59 pyrimidine joined
MasterDuke hadn't had to change 8119, just changed all the [] to atpos 22:59
well, i've now changed my changes a couples times, so i don't think so, but couldn't swear to it 23:00
jnthn Hm, I@m surprised it didn't have to become nqp::atpos_s(@names, 0) there
MasterDuke oh sorry, my line #s were off. yes, that became an apos 23:01
jnthn :) 23:06
MasterDuke ok, from 15,335 BOOTStrs to 14,883. now to see how it spectests 23:20

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!