samcv note to self: skin calming bodywish stings horribly if you accidently get in your eyes 00:00
i feel they should have put a warning label on it.
timotimo you get what you wish for, eh?
samcv "Skin calming eye irritant body wash" 00:01
i'm sure it would sell millions
MasterDuke the "allocated" tab: i.imgur.com/zgZAAGF.png 00:02
timotimo i wonder if it, like, endlessly recurses or something? 00:03
samcv heaptrack is good yes MasterDuke ? how should i compile mvm to get best diagnostic stuff on it.
just full debug?
MasterDuke i usually just do --debug=3 00:04
hm, though i think i'm still using the version i compiled with --optimize=1
timotimo: would that effect the profile? 00:05
samcv: yeah, i've liked it
timotimo shouldn't
heaptrack most likely just instruments malloc and free and such
MasterDuke believe so
timotimo those are cross-.so-file
so the compiler wouldn't muck them up 00:06
samcv wow my gh-pages branch is already really big. i'm trying to check it out now so i can add a page to it 00:09
it's still unpacking objects after 30 secs
well not really meant for anybody to check out anyway. 00:10
timotimo i wonder if you should force-push over it a bunch so that it frees up space 00:11
or instead keep the history because history is good?
samcv history is good 00:12
idk i may force push over it if it gets too big
timotimo right
samcv but nobody is supposed to check out that branch
timotimo can you "git clone" without grabbing the extra branch?
samcv it just really keeps history. the main branch has never had any appimages in it
yeah
i don't think it grabs it
timotimo i know you can --depth=1, but that leaves you with a repo that can't do stuff
samcv i should try though
MasterDuke if anybody is interested, profile of `htmlify.pl --no-highlight --sparse=50`: gist.github.com/MasterDuke17/754d8...9516c87ae3 00:13
samcv: what's gh-pages?
timotimo aaargh the extremely long filenames :D
samcv it can host websites
i'm gonna set up a github pages for appimages and also one for moarvm coverage reports 00:14
MasterDuke ah, right. very cool
timotimo aha, it spends a whole lot of time inside "wait"
samcv so we'll get automated mvm coverage reports and i won't have to do any work
timotimo wait, is that without --parallel?
samcv well. after it's automated
timotimo like, it doesn't highlight, so what is it waiting for?
MasterDuke --parallel=1
samcv ok yeah it's cloning everything when i ask to clone it 00:15
MasterDuke i.e., the default
timotimo that could be harsh
if every appimage is 10 megs big 00:16
and you do daily blead and master builds 00:17
MasterDuke if i remember how i originally did it (though it's been changed since then by others), --parallel is just how many things to `start` at once
timotimo right 00:18
oh, so the main thread probably awaits all-of the started things
MasterDuke so --parallel=1 is essentially `start` and then immediately `await`
timotimo right
in that case we'll get like no info out of the profile
since it doesn't know how to reach into other threads' data yet
MasterDuke i do remember when i first did it, that with --parallel=10, only 1 in 5 runs would finish, but running it 5 times was still faster than once non-parallelized 00:20
timotimo the problem with that is that the moment we reach "end of profile, collect data now", all the other threads are like "yup, still runnin'!"
MasterDuke so think you can knock out making the profiler thread-aware tonight? 00:21
that's what, a good 10-15min job?
samcv git clone --single-branch github.com/samcv/rakudo-appimage.git # will only clone the master branch
and is super fast
so put a readme
timotimo yes, readme be good 00:22
i gotta go sleep now
have a good one!
oh
samcv in the future, sleep is illegal
timotimo how do you feel about using the appimages to travis-test our ecosystem modules?
samcv sure 00:23
timotimo thought so!
it should save boatloads of time
samcv automate everything
and can have it autocommit reports too now that i've mastered that 00:24
though i still have no clue how gh-pages works
timotimo well, you put the right files in there an dit ends up on like samcv.github.io or something
samcv yep 00:26
samcv.github.io/rakudo-appimage/ ok well i got it sorta working 00:27
timotimo ah, yes, good 00:28
i believe gh-pages can also run some static-website-generators for you 00:29
but that may be outdated knowledge
okay, bedtime for reals
MasterDuke later 00:30
samcv well i got this samcv.github.io/rakudo-appimage/ 01:03
so that's nice
got it generating a basic inde
MasterDuke what's star-testing? 01:36
samcv it's rakudo star. the testing refers to the appimage thing 01:46
makes sense if you compare the source of perl6-stable and perl6-testing here github.com/samcv/rakudo-appimage
the testing one resolves relative paths and such
the other one thinks paths that are relative are relative to the appimages virtual disk
MasterDuke ah, ok
samcv .o(if we build an appimage with EVERY eco module then who cares if you can't install them) 01:47
heh
brb walk
gonna walk the puppy
though i'm about to change testing to stable i think. becuase i can't run any of the rakudo sanity tests 02:02
without it
gonna go ahead and remove the appimages that were made before testing had been implemented so people don't dl borked files (cuase it only updates files if they pass a travis build) but before they passed even if it didn't properly work 02:20
[Tux] This is Rakudo version 2017.03-222-g184d49996 built on MoarVM version 2017.03-128-gc9ab59c6 05:52
csv-ip5xs 3.028
test 12.611
test-t 5.160 - 5.172
csv-parser 12.874
samcv .tell Zoffix HTML::Parser is broken due to IO changes 08:54
yoleaux samcv: I'll pass your message to Zoffix.
samcv actually haven't updated my perl6 in a day or two gonna do that 08:55
err wait it's zef doing it 09:14
lizmat afk for the coming day or so& 09:34
samcv Zoffix, ok it's the XML module that's failing Calling from-xml-stream(IO::Handle) will never work with declared signature (IO $input) on latest rakudo 10:01
weird and i get that error if i do `echo hi | perl6` too 10:03
odd
echo "say 'welcome';" | perl6 # this shows the error. but this -> echo "say 'welcome'; exit;" | perl6 #this is fine 10:04
pmurias why does: sub infix:<++> {'this creates %?LANG'}; CALLER::<%?LANG>; work? 10:21
the %?LANG doesn't seem to be installed as dynamic
m: sub infix:<++> {...}; say(VAR(%?LANG).dynamic) 10:22
camelia Nil
geekosaur it's "dynamic" but at compile time; by run time it's presumably no longer dynamic 10:25
by run time it's too late to change the language the code was compiled with 10:26
at one point there was a bug allowing ?-twigil vars to be created uselessly at run time; if you managed to do so, maybe you found a remaining edge case? 10:28
samcv night all 10:33
pmurias night
geekosaur: dynamic refers the ability to get the variable with CALLER, not to overwrite it 10:34
geekosaur that's not the point. the point is compile time. the caller nevertheless has a language it was compiled in, and presumably the compile-time $?LANG is therefore available for it 10:35
it's not *runtime*-dynamic, except in the sense that every scope has a language it was compiled with and you can introspect that scope for that language
pmurias geekosaur: doesn't dynamic refer to the ability to introspect it from the caller (which EVAL depends on) 10:53
geekosaur: i'm trying to figure out what's wrong on the js backend that CALLER::<%?LANG> doesn't work
Zoffix . 10:54
yoleaux 08:54Z <samcv> Zoffix: HTML::Parser is broken due to IO changes
geekosaur afaik dynamic means * twigil
Zoffix XML has been already fixed 2 days ago. Update it.
samcv the version was not bumped 10:55
can can --force install it. i'll open an issue 10:56
Zoffix, i made sure i ran zef upgrade. but people don't always bump versions :\ 10:58
timotimo pmurias: i think geekosaur didn't mean "dynamic" as in "perl6 dynamic variable", just "based on the callstack, not on lexical environments"
geekosaur I was walking around that (hence scare quotes on "dynamic") 10:59
but yes, it's confusing 11:00
and because it's a compile time thing, it's likely not going to obey simple rules
timotimo the value of %?LANG is dynamic in the compiler because the compiler has a call stack that resembles the lexical scopes of the program it's compiling 11:01
samcv and looks like even --force isn't working. it's using the already cached files in ~/.zef/store
eek
timotimo there's "zef nuke" :P 11:11
pmurias n
sorry
Zoffix zef --/cache --force install XML 11:27
Zoffix nukes everything on every upgrade
update-perl6 is aliased to `rm -fr ~/.zef; rm -fr ~/.perl6; rm -fr ~/.rakudobrew/; git clone github.com/tadzik/rakudobrew ~/.rakudobrew; rakudobrew build moar; rakudobrew build zef; zef --/test install ā€¦ long list of modules I use ā€¦
Zoffix adds --serial to zef switches in that command... 11:28
pmurias (try ...)-- 11:40
Zoffix ZofBot: am I getting fired? 11:46
ZofBot Zoffix, On the other hand, pragmas designed to influence deep semantics should not be named identically, though of course some similarity is good
Zoffix Man, I still don't have access to Adobe's Creative Suite at work. 11:47
TFW you can't figure out if someone's amazingly incompetent to not renew a license or you're getting canned so they didn't bother renewing it :S 11:48
Zoffix looks at .slurp-rest impl 11:56
takes :bin and :enc...
with a where clause checking :bin is true in one multi 11:59
pmurias why does Scalar.dynamic returns False, True *and* Nil, Nil being treated as not dynamic by the optimizer and as dynamic by the caller? 12:00
s/caller/CALLER 12:01
Zoffix On replacement .slurp(), don't wanna do that. Going to use $!encoding to figure out mode to use. Keeps it all the same for all other routines.
pmurias CALLER:: uses the "does not give a f**k" form of try so it might be treating it that way accidently 12:02
Zoffix What form is that?
pmurias try without a handler 12:16
Zoffix ===SORRY!=== 13:11
arg expression cannot be void
Well, that's new :|
MasterDuke i've never seen that before either 13:12
Zoffix was due to dd in dd nqp::seekfh($!PIO, $size, 0) 13:13
say works there too
MasterDuke happens in src/vm/moar/QAST/QASTOperationsMAST.nqp:1333 13:14
timotimo ah, yeah, seekfh doesn't return anything 13:36
that seems to upset the compiler, though the message could definitely be less LTA 13:37
MasterDuke `nqp::die("arg expression cannot be void") if $arg_mast.result_kind == $MVM_reg_void;` 13:38
what could it do instead of die?
timotimo does it point anywhere near where the problem happened? 13:40
can i get at the JSONPrettyGrammar inside the rakudo internals? 13:41
like, from inside user level perl6 code?
it's probably internal to that. .. bleh, need to use something else,t hen 13:44
Zoffix_ I'm a genius! 13:45
Wrote .slurp that's ~16x faster... but the genius part is I wrote it in /tmp/z.p6 file and then tried to slurp a very huge file, thereby "forkbombing" my box. 13:46
It looks frozen solid, but I don't wanna reboot it 'cause I'll lose that file :P
MasterDuke can you ssh in to it? 13:48
Zoffix_ Doubt it. I don't think it even has ssh server installed. (It's a local VirtualBox VM) 13:49
There's not much code lost and I remember it from memory, I'm just amused by how I lost it :)
MasterDuke wonder how easy it is to browse virtualbox's disk images. i.e., just close it saving state and then access the fs inside the image 13:51
timotimo boot a livecd into the box perhaps? 13:52
if the /tmp gets freed at startup
if /tmp is a memdisk, however ...
MasterDuke timotimo: i can repro it locally, but it doesn't say anything about where in my script there's a problem 13:59
timotimo right, you think we can figure that out from there? do we have a .node somewhere in the vicinity? 14:00
MasterDuke `$arg, $qastcomp, @ins, @arg_regs, @arg_flags, @arg_kinds` are args to handle_arg. `$arg_mast := $qastcomp.as_mast($arg), my int $arg_mast_kind := $arg_mast.result_kind;` are the only variables created before it dies 14:02
timotimo hm. i wonder how other places in there make error messages ... if they do at all? 14:11
MasterDuke nqp::die("arg code did not result in a MAST::Local") unless $arg_mast.result_reg && $arg_mast.result_reg ~~ MAST::Local;
is the very next line
but that's the only other die in that functino
timotimo maybe we should stash this for later 14:12
MasterDuke *function
timotimo though maybe the $qastcomp has something for us? not sure what $arg exactly would be
MasterDuke if you're in the middle of making grammars faster don't get distracted by this, i'll play around a bit 14:13
timotimo sadly, it won't make grammars much faster all in all
the improvement won't scale, it's just a tiny bit off of startup time
MasterDuke improving startup time is awesome, it makes everything else feel faster 14:14
timotimo and it only cuts off of the startup time when startup includes running a user-made grammar
so regular rakudo startup is not going to be changed
MasterDuke well, one of the perl 6 selling points is custom grammars, and #perl6 certainly sees a lot of questions about them, so it's a good optimization 14:17
timotimo i guess it's fair 14:18
Geth rakudo/nom: 064b5858b2 | (Timo Paulssen)++ | 3 files
allow Grammars to precompute their nfas during precompilation
 14:30
MasterDuke nice 14:34
timotimo well, if you load a precompiled module that has a grammar in it, like, a million times ... i'm sure it'll save you a second or two! 14:38
MasterDuke do the savings depend on the size of the grammar? 14:39
jnthn Hm, I should try it against the YAMLish profile I took
timotimo depends on the amount of stuff in the grammar that you touch
though "touching" in this sense can be very contagious and far-reaching 14:40
jnthn According to the profile I saw, we spent a ton of time in merge_substates or related when parsing a small (just 5-10 lines) YAML file. It was like 50% of the parse time.
I'll reproduce that and then upgrade Rakudo :) 14:41
timotimo 50% of 10ms is still just 5ms :D
MasterDuke would this help Text::CSV since it uses Slang::Tuxic? or Perl6::Parser?
timotimo hm, not quite sure 14:42
Zoffix Killed my VM again, but now I'm surprised. It's got two loops of for ^$x { my $fh = $s.open; $ = $fh.slurp-rest; $fh.close } slurping a 6MB file 300 times... 14:54
I would've figured the $ would be reused and previous slurpage cleaned up? 14:55
jnthn That isn't how scalars in Perl 6 work 14:56
The Scalar stores a reference to something
The something is the expensive thing
There's probably some missing memory pressure somewhere
If it's being overly hungry
Zoffix Oh, so all those slurps still live in memory until gc 14:57
OK. Now I'm getting sane numbers again :) 14:59
k, my new IO::Handle.slurp in binary mode is 16x faster and uses 2.1x less memory 15:05
ZofBot: but will it blend?
ZofBot Zoffix, The mark characters are ignored only for the purpose of determining the truth of the assertion; the actual text matched includes all ignored characters, including any that follow the final base character
jnthn Zoffix: Wow, looking forward to seeing that patch :) 15:06
Zoffix I still have to run stresstest on it :}
timotimo i wonder what we did bad in slurps binary mode 15:07
jnthn Oh dear, bumping Rakudo version doesn't end too well for $dayjob project 15:10
Zoffix oops
jnthn No such method 'abspath' for invocant of type 'IO::Path' in block at /home/jnthn/dev/MoarVM/install/share/perl6/site/sources/8922A2BA7BE89CE9C1AF2017551EC8B1435BA924 (Digest::SHA1::Native) line 5
Zoffix jnthn: change to .absolute
jnthn: oh, upgrade the module 15:11
It was fixed yesterday
jnthn Oh, cool :)
Zoffix huggable: IO kills
huggable Zoffix, 15 x [āœ”]; 3 x [āœ˜]: See gist.github.com/zoffixznet/0ea1f4d...3f8dd11cc1
jnthn Doing so :)
Zoffix um... zef --force install Digest::SHA1::Native
'cause the version wasn't bumped.
Zoffix notes to that for any future PRs :|
jnthn Aww, the version wasn't bumped
Zoffix hopes that's the only breakage :} 15:12
jnthn The test suite thinks so :) 15:25
Need to check it on Windows also
MasterDuke `$arg_mast := $qastcomp.coerce($arg_mast, $MVM_reg_str) if $arg_mast.result_kind == $MVM_reg_void;` makes `./nqp-m -e 'say(nqp::seekfh(nqp::open("a", "r"), 1, 0))'` just print an empty string instead of dying
and passes spectest. but it seems a bit hackish... 15:26
jnthn That loosk...weird
MasterDuke before you'd get: `===SORRY!===arg expression cannot be void` 15:27
jnthn That seems more sensible.
MasterDuke then any way to get some information about where things went wrong into the message? 15:28
Zoffix buggable: speed 2
buggable Zoffix, ā–ā– data for 2017-04-12ā€“2017-04-12; range: 5.160sā€“5.172s; ~0% difference
Zoffix buggable: speed 3
buggable Zoffix, ā–ā–ƒā–ƒ data for 2017-04-11ā€“2017-04-12; range: 4.996sā€“5.172s; 4% slower
Zoffix buggable: speed 4
buggable Zoffix, ā–ā–‚ā–„ā–„ data for 2017-04-11ā€“2017-04-12; range: 4.942sā€“5.172s; 5% slower
Zoffix buggable: speed 6
buggable Zoffix, ā–ƒā–ā–ā–‚ā–„ā–… data for 2017-04-10ā€“2017-04-12; range: 4.921sā€“5.172s; 3% slower
Zoffix The 5% slower seems to be from the MoarVM bump last night: github.com/rakudo/rakudo/commit/d0924f1a28 15:29
Unless it's something with [Tux]'s box that made the measurements this morning slower
jnthn MasterDuke: Probably, though I thought we had various CATCH blocks in place that annotated errors like that with a source location
pmurias jnthn: what should Scalar.dynamic return when the scalar has no descriptor? It currently return Nil in this case but I think it should be either True or False 15:30
jnthn pmurias: False, I'd think
MasterDuke jnthn: fyi, this is in src/vm/moar/QAST/QASTOperationsMAST.nqp, what i commented out is `nqp::die("arg expression cannot be void")` 15:31
jnthn Well, the error is correct
pmurias ok. so I should explicitly set %?LANG as dynamic? 15:32
jnthn If you use nqp::ops that are void you should expect a code-gen error if you try to use them. It'd be good if it could give the nearest annotated source location, however. 15:33
Zoffix MasterDuke: FYI, it took me exactly 3 seconds to debug that error in my code
jnthn But I thought that we had a CATCH in stmts notes that would add location into and rethrow 15:34
Zoffix s/FYI/FWIW/
jnthn pmurias: Ah, 'cus EVAL needs to access it through the caller chain?
pmurias: Seems reasonable enough 15:35
MasterDuke Zoffix: yeah, i wasn't going to PR my change, but like jnthn said, getting the nearest annotated source location would be nice
pmurias jnthn: yes, EVAL needs that. Should other compile-time variables be marked as dynamic too? 15:36
jnthn I was wondering that, but don't have an immediate feeling for it :) 15:37
Probably not all though
Geth nqp/uncurse: bd4f38a416 | TimToady++ | src/QRegex/Cursor.nqp
PRECURSOR should not trim $!orig
jnthn Because some are actually just transforms
MasterDuke jnthn: fwiw, this is the relevant ast gist.github.com/MasterDuke17/c3608...69bc7faa72 15:39
TimToady this might or might not be relevant, but $?LANG is now supposed to be dynamically generated at the point of mention like $?LINE, but as far as I know EVAL doesn't use it quite yet 15:44
TimToady goes to get another cuppa to see if his brane unfogs anysome
Zoffix :o It took me thiiis long to realize I can write LĀ«C<Type>|/type/TypeĀ» in docs as just L<Type> :| 15:46
TimToady didn't sleep so well last night, between having the snuffles and having the house full of butyl mercaptans all night
Zoffix I even bound LĀ«C<>|/type/Ā» to a key on my keyboard :(
TimToady courtesy of a skunk in the back yard that probably met a cat or a racoon
Zoffix ewww 15:47
Zoffix remembers moving to US and smelling the first skunk roadkill.
TimToady progress report on uncurse, we're now down to blowing only 25 spectest .t files completely out of the water :)
Zoffix There were moments I thought I was going to die.
Cool \o/ 15:48
jnthn TimToady++ 15:51
TimToady and it does seem slightly faster, but the big speedups will come later
the problem with our current cstack is that it's more optimized for backtracking than for forwardtracking
I think we can make parsing a lot faster by making backtracking a bit slower 15:52
well, hopefully without slowing down backtracking, but we'll see
but basically by assuming we will succeed rather than assuming we'll fail 15:53
jnthn Sounds interesting :)
TimToady and by putting things directly into where the match object can provide them
another potential speedup, once we get $<foo> to not go through the hash interface mandatorily, is to avoid capturing to hashes at all 15:54
for any given rule, the names we capture are statically known, so could just be a static mapping to fixed array locations 15:55
jnthn Indeed
I guess the challenge there is that we cheat like heck on hash/array access
TimToady a capturing match really only needs to know "put yourself into this array at this position if you match"
jnthn In NQP 15:56
We can stop doing that
TimToady and that array will be in the cursor/match object itself, not the cstack, so we never have to copy
yes, we can either implement AT-KEY, but that still is a string interface on each call 15:57
or we can do something maybe a little smarter at compile time, or on first use, or somethin'
AT-KEYish stuff might be sufficient with smart enough type spesh
but currently the $/<foo> interface is boxing us into a performance corner I don't like 15:58
jnthn Well, it was an optimization with the assumption we'd have a real array and real hash
TimToady I'm thinking AT-KEY is the slow path
and the compile can be smarter about literal $<foo> 15:59
jnthn That dates back to the Parrot days
TimToady sure, not complaining about how we go here :)
*got
jnthn And, to be fair, the early Moar days too when we had no inlining
TimToady just trying to see around the corners to where we'd like to be
jnthn Now it's mostly just more complexity in the VM :)
TimToady we don't internal hash keys yet the way P5 does, do we? 16:01
*intern
jnthn We intern strings inside a single compilation unit
And we have a fast path in hash string equality checks that first looks at the pointer 16:02
We don't yet do interning between compilation units
And don't do anything about non-literal strings
TimToady that doesn't help one mention of $<foo> vs another $<foo> which is presumably a different string "foo"
or does it? 16:03
timotimo if they are in the same file, i'd imagine they get the same "foo" object
when it lands in the string hash they'll be deduplicated
TimToady okay
jnthn Yeah, what timotimo said. We de-dupe at bytecode assembly time
I suspect that there'd be a bit of a win to be had if we de-duped cross-compilation unit 16:04
timotimo it'd be interesting to see if we can "just" take strings from another compunit when we load one from another's string heap, but then we'll need a hash again, and then thread-safety and urgh
jnthn Because then your grammar and actions, if in separate files, could also get the faster path
But of course you gotta then be mighty careful that you don't introduce a new memory leak for EVAL :) 16:05
timotimo hm, is our string heap sorted? i seem to remember it is
so we'd have the ability to use merging of pre-sorted lists here, or perhaps binary search.
TimToady anyway, that's all well and good, but I think we can do better by taking more advantage of the staticness of rule names directly, and basically have a string mapper to position for each rule
timotimo right
TimToady and mostly make that translation instantly at compile time 16:06
timotimo can't imagine right now what corners the data has to flow around there :)
TimToady anyhoo that's the direction I'm thinking once the uncurse branch settles down
timotimo sounds exciting
TimToady just not calling .CURSOR and .MATCH all over the place should help some too 16:07
jnthn It'll work well inside of rules
Action methods I guess will still have to do a hash lookup along the way
TimToady but hopefully not more than one lookup per call, anyway 16:08
and we can also look at gluing action routines more closely to rules at compile time or first use time
the double dispatch seems suboptimal 16:09
though we do have to do both dispatches at least once, since there's no 1-to-1 mapping necessarily
timotimo reminds me a little of how OpenGL did vertex and fragment shaders 16:10
you were forced to compile a new object for every combination of vertex and fragment shader
even though the interface between the two is kind of static
there's extensions for more freedom there, though
jnthn I've already got some plans to change how we call action methods
So we can better spesh 'em
And even inline the tiny ones
TimToady of course, if we really want to parse Perl 6 fast, we write a yacc grammar that parses "standard" P6 and just drops down into recursive descent when the language changes :) 16:11
timotimo that sounds nice
is that based on more code gen? like CompilationHelpers?
timotimo grocery shopping 16:14
TimToady one thing I noticed was that the action method actually seems to currently be in the dynamic scope of its corresponding rule, which is nice, but I hadn't expected 16:17
for instance, rule O sets %*SPEC, and action method O uses that, which wouldn't work if the action method were called at the same level as the rule
anyway, we probably need to preserve that
however we hook rules with actions 16:18
TimToady turns his glare upon the 25 broken .t files... 16:19
Zoffix *files magically stop being broken* 16:20
jnthn TimToady: I'd figured it had to be that way :) 16:21
'cus of the dynamic variable pattern you just mentioned :)
TimToady well, it fits with how we were thinking of action methods as equivalent to embedded {}
jnthn Aye 16:22
I don't think it's a perf issue to have it this way
TimToady so adding an action method to / foo | bar / is turning it into / [ foo | bar ] { your ac here } /
jnthn If anything, it's good if we fix our code-gen
Because we can emit the action callsite directly in the regex body
Meaning that it'll be speshable, inlinable, etc. 16:23
Zoffix Ah. There's no apparent memory improvement with my .slurp patch. When I run it on my VM with 24GB, the use is the same. I guess the diff I've seen was due to GCed stuff on my 4GB VM on homebox
The speed is there tho :P
TimToady most of P5 trades *more* memory for more speed :)
so keeping it the same is kind of a win, or at least a non-lose :) 16:24
Zoffix hehe
TimToady me puts too many smileys on his messages :)
jnthn Me too :D 16:26
Zoffix :D :D :D
Booo. My faster slurp crashed and burned in stresstesting. All is no lost however, I should still be able to reap the benefit. But will have to do it after work. 16:35
jnthn Zoffix: Dammit. Turns out that on Windows, various things use .abspath to locate DLLs :( 16:42
So there's more module breakage there
Zoffix :(
Module as in ecosystem module? 16:43
All but three are fixed
huggable: IO kills
huggable Zoffix, 15 x [āœ”]; 3 x [āœ˜]: See gist.github.com/zoffixznet/0ea1f4d...3f8dd11cc1
jnthn Yeah
Oh, maybe something didn't version bump
Zoffix p6-native-libc is listed as unfixed. There's a PR for it tho: github.com/cygx/p6-native-libc/pull/3 16:44
And other two are Crust and Net::FTP
jnthn Archive::LibArchive::Raw seems not updated 16:45
$ git grep abspath
lib/Archive/Libarchive/Raw.pm6: ?? %?RESOURCES<libarchive.dll>.abspath
(that's after a git pull)
oh, hang on
haha
No, that's my local fork of it 16:46
Zoffix Is there a way to tell if a handle can't bee nqp::seekfh'ed other than a try?
jnthn Hm, it had been version bumped 16:47
OK, I dunno what I did...
But looks better now
Zoffix: In MoarVM I guess we know... 16:48
Oh, but maybe not even reliably then
Yeah, I think probably not
Zoffix Alright
jnthn I was thinking we could check if the handle type implemented the seekable thing
(But then realized that doesn't mean it will succeed) 16:50
Zoffix "/* Cannot seek a TTY of named pipe (could fake the forward case, probably). */"
Is it a safe assumption in Rakudo that if I can forward-seek a handle, I can seek back to original position too? 16:51
'cause that's what my 16x improvement is doing. And if it ain't safe, then it's out of the window 16:52
And it fails with $*IN, so Imma pop a `try .seek` on it and fallback to the slowpath
jnthn Heh, not if we do what that comment suggests...
What was your speedup?
timotimo oh
probably seek + tell + pre-size
jnthn aha 16:53
Zoffix jnthn: 16x
And 2x for a "few bytes" file
jnthn No, I meant what did you do to acehive it :)
geekosaur also can't seek a socket
or a normal pipe
Zoffix jnthn: the tell -> seek to end -> tell -> seek back; use the diff 16:54
jnthn: gist.github.com/zoffixznet/310c12c...bc3ff68787
timotimo geekosaur: yeah, i have trouble finding my socks, too
geekosaur also, is there a binding to fstat? 16:55
Zoffix And the code in Pipe.pm in that diff, will be moved back into Handle.pm and be executed when `try seek` fails. 16:56
jnthn Zoffix: Can we not just pass some integer max value to read?
And it'll read all it can and return to us? 16:57
That'd save the whole seek/tell thing
Zoffix jnthn: good plan.
Gonna try that after work. 16:58
Um... lol. There are 2 plumbers. One came in and said he'd be back, I told him to just come in. I'm sitting in my bedroom right now with the door closed and second plumber came in. And judging by the amount of swearing he's doing he thinks no one's home :) 17:02
brb... gonna scare teh crap outta him :)
heh 17:04
Well. That was good 3 seconds of fun :)
timotimo :D 17:05
TimToady
.oO(Mr Plumber, in the kitchen, with a lead pipe.)
 17:08
Zoffix No idea what handle->body.ops->sync_readable->read_bytes(tc, handle, &buf, length); calls (like where that ->read_bytes is at) 17:12
But thinking: wouldn't passing a max int allocate a crapton of memory to read all of the stuff? 17:13
Yup 17:15
$ perl6 -e 'use nqp; dd nqp::readfh(nqp::getattr("rakudo/foo".IO.open(:bin), IO::Handle, q|$!PIO|), buf8.new, 2**63-1)'
MoarVM panic: Memory allocation failed; could not allocate 18446744073709551615 bytes
jnthn: I guess we could make it a bit larger than 0x100000 but probably not by much 17:17
m: say 0x100000/1024
camelia 1024
Zoffix 1mb
(context: we currently read in 0x100000 chunks, .appending to a buf each time) 17:19
I guess I can knock it all on the head. If you want fast file slurp: use IO::Path.slurp, not IO::Handle. 17:20
Zoffix chooses that option
bartolin Zoffix: I plan to open a ticket for newly failing IO related tests on JVM. I'd like to fudge those tests for now. (If I'm able to make them work, I'll do that instead.) If you have a better suggestion please let me know 17:49
Zoffix bartolin: other than leaving them as is and letting me take care of them on Monday, no suggestions. 17:58
bartolin Zoffix: ok, I'm going to fudge them. Maybe that makes things a bit easier for you, too. 18:04
Geth rakudo/nom: f1b4af7af4 | (Zoffix Znet)++ | src/core/IO/Handle.pm
[io grant] Implement IO::Handle.slurp

  - Per IO plan, add .slurp as a replacement for .slurp-rest
   - Will add a DEPRECATED to .slurp-rest when we cut 6.d, as
   currently the compiler version for that is unknown
  - Leave .slurp-rest as is, but in new .slurp, toss :bin and :enc
   params. Use the handle's $!encoding attr. This aligns the routine
... (8 more lines)
roast: 7e4a2ae34b | (Zoffix Znet)++ | 15 files
[io grant] Swap .slurp-rest to .slurp

Rakudo impl: github.com/rakudo/rakudo/commit/f1b4af7af4
Since current master is a to-be 6.d and .slurp-rest is to be deprecated in 6.d, we piggy-back on existing tests, but just swapping to the new method & changing 2 tests that use :bin mode without bin-modded handle
The .slurp-rest() tests will still exist in 6.c-errata branch and will work just fine with current and 6.d rakudo; issuing a deprecation warning in 6.d;		 18:08
Zoffix I figured that's OK ^ . Releases do 6.c-errata tests and once I have the bot running, it'll run them daily. So .slurp-rest is still tested as before if you consider all the branches of tests in combination. 18:10
we still have .tree in Any 18:16
m: <a b c>.tree.say
camelia (a b c)
Zoffix m: (<a b c>, <z b g>).tree.say 18:17
camelia ((a b c) (z b g))
Zoffix doesn't know what its point is
TimToady it's supposed to take mapping arguments for each level of the tree 19:11
Zoffix m: .signature.say for ().^lookup("tree").candidates 19:12
camelia (Any:U $: *%_)
(Any:D $: *%_)
(Any:D $: Whatever, *%_)
(Any:D $: Cool $count, *%_)
(Any:D $: @ (&first, *@rest), *%_)
(Any:D $: &first, *@rest, *%_)
Zoffix m: say (<a b c>, <z b g>).tree: &uc 19:13
camelia A B C Z B G
Zoffix m: say (<a b c>, <z b g>).tree: {"[$_]"} 19:14
camelia [a b c z b g]
Zoffix shrugs
TimToady it was only ever half implemented the way it was specced
Zoffix Ah
m: printf '%.3fe%d', $_/10**(.chars-1), .chars-1 with 123456789 20:01
camelia 1.235e8
Zoffix m: printf '%.3fe%d', $_/10**(.chars-1), .chars-1 with [*] 1..361
camelia 1.438e768
Zoffix FFS 20:43
My plan to power through the IO stuff before release is being stymied by a couple of dirty strange men occupying area where I wanna take a shit.
I'm actually working for my other job now--I'm not even supposed to. Can't think straight will all the banging 20:44
ZofBot: HELP ME!
ZofBot Zoffix, pred
jnthn Hurrah, seems I've nailed the occasional data loss bug in IO::Socket::Async::SSL 21:00
That means it's pretty close to ready to ship off to the ecosystem 21:01
Zoffix Sweet 21:03
MasterDuke jnthn: i can't get any of the nqp::die() conditions in src/vm/moar/QAST/QASTOperationsMAST.nqp to give source location. e.g., 21:18
m: use nqp; nqp::if_i("foo")
camelia ===SORRY!===
No registered operation handler for 'if_i'
MasterDuke m: use nqp; nqp::die("foo")
camelia foo
in block <unit> at <tmp> line 1
jnthn MasterDuke: Hmm, maybe there is no current handling for that, then 21:21
The most common one is no such operation handler, which is easy to grep for :)
And, to be fair, errors at this layer are pretty rare :) 21:22
MasterDuke true. but if it's easy to fix, seems like a good thing to do. i just don't know where the fix would go 21:23
[Coke] fff 21:33
MasterDuke jnthn, Zoffix: a lot of the nqp::die()s in src/vm/moar/QAST/QASTOperationsMAST.nqp could at least have some extra information added (e.g., the name of the void arg in the case Zoffix found), like how the `No registered operation handler for '<foo>'` one does 21:43
think it would be worth adding relevant info where possible? 21:44
Zoffix My opinion is: not if there's any sort of performance impact 21:45
MasterDuke in pretty much every case we're already dying, so i doubt an extra nqp::elems, or .op() would really matter 21:46
jnthn If we're dying anyway it's likely quite cheap 21:47
Zoffix Ah right.
I mean if there's any perf impact for non-dying paths :)
MasterDuke but i wasn't planning on doing anything extensive, just there are a couple things like "can only have one foo", and i'd just add ", got '+@foo'" 21:48
Zoffix Though even with dying anyway: we have 87 CATCHes just in core alone; people catch deaths
This is dying in nqp tho? Never mind me. 21:49
Zoffix closes the chat window and focuses on stuff :)
MasterDuke it'll be a PR so (any|every)body can take a look, but i'm just anticipating minor changes 21:51
jnthn I'd in principle be fine with such a PR :) 21:54
Zoffix Interesting. My IO::Path.slurp is dying with "Failed to close filehandle: bad file descriptor" after N number of iterations, where the number of iterations varies with env vars I specify... Like bogus env vars, but just adding one changes at which iterations it throws that :S 22:11
ZofBot: what sourcery is this 22:12
ZofBot Zoffix, it's still unpacking objects after 30 secs
timotimo interesting, can you strace that?
Zoffix I've no idea what that means... just run strace perl6 foo.p6?
timotimo yeah
thatg ives you a trace of all system calls
you ought to be able to see what file descriptor number it complains about by 1) when the error message from rakudo comes, and 2) when something is listed as = ESOMEERROR 22:13
Zoffix there's like a billion screenfuls of output
timotimo yup 22:14
mostly mmap and such
i know you can filter with -e, but you'll need a negative filter
Zoffix Descriptor 2? gist.github.com/zoffixznet/8423293...51e0e32c79 22:16
timotimo no, tha tought to be stderr 22:19
where it outputs the error
Zoffix Here's the code: gist.github.com/zoffixznet/344a158...3e744675ce
timotimo close(-1) = -1 EBADF (Bad file descriptor)
it's trying to close fd -1, that's very wrong
Zoffix ugh --optimize=off gets rid of the bug :/ 22:21
Sad, 'cause it means it's likely not in my code :\ 22:22
timotimo uh oh
it doesn't panic, so you should be able to --ll-exception perhaps?
Zoffix huh 22:23
"at SETTING::src/core/IO/Handle.pm:878 (./CORE.setting.moarvm:DESTROY)"
timotimo now that's interesting 22:24
Zoffix Ah, I gotta unset the $!PIO in the handle, I'm guessing
timotimo perhaps it's already closed and closing it manually has reset or invalidated the fd?
and the DESTROY method is erroneously running?
Zoffix I see my mistake
timotimo good! 22:25
Zoffix Old code called $handle.close, but I'm doing nqp::closefh($PIO)
Thanks :)
timotimo ah
sure thing
i'm really glad to be of help 22:27
Geth rakudo/nom: c13480c82a | (Zoffix Znet)++ | src/core/IO/Path.pm
[io grant] IO::Path.slurp: make 12%-35% faster; propagate Failures

  - In fast path: use `try` so we catch throwage by nqp::open
   - and if we threw there, the slowpath will return a Failure
  - The perf improvent is only for the slurp call itself, not the actual
   reading, so stated measurements apply only tiny files		 22:59
Zoffix *to slurps of tiny files 23:02
Is there a full list of encodings MoarVM supports? 23:31
Like, stuff you can give to open :enc<...>
Is this it? github.com/MoarVM/MoarVM/blob/mast...1940-L1951 23:32
Looks like it 23:33
timotimo we do have that "canonicalize_encoding" function or table or somesuch 23:35
Zoffix R::I.NORMALIZE_ENCODING
timotimo btw, if you find us a junior coder, they could try putting in all the iso-9999-blah encodings
into moarvm
there's a table for every one of those on the unicode 'site somewhere
Zoffix Cool
Maybe I take a crack at it later on 23:36
timotimo just in the last month or maybe two it was discussed, but i don't remember who was involved in the conversation
i'm not even sure if it was in #moarvm or not
8859 is the number i was looking for 23:37
www.unicode.org/Public/MAPPINGS/ISO8859/ 23:38
irclog.perlgeek.de/perl6/2017-03-24#i_14318239 - and surrounding lines
Zoffix huggable: encoding 23:39
huggable Zoffix, nothing found
Zoffix huggable: encoding :is: Add more encodings to MoarVM: irclog.perlgeek.de/perl6/2017-03-24#i_14318239
huggable Zoffix, Added encoding as Add more encodings to MoarVM: irclog.perlgeek.de/perl6/2017-03-24#i_14318239
timotimo did you ever wonder what a timeline where EBCDIC won over ASCII would look like? 23:42
Zoffix Nope 23:43
timotimo OK!
Zoffix I don't even know what EBCDIC is about; it's one of those "before I was born" things
timotimo www.unicode.org/Public/MAPPINGS/VEN...sions.html *cough* 23:44
www.unicode.org/Public/MAPPINGS/VEN...FT/EBCDIC/ - on top of that
because of course there's codemaps on top of ecbdic
cp037_IBMUSCanada - don't you just love that
Zoffix heh, well, looking at one of those files, I see why it didn't win. The letter ranges are not contiguious 23:45
timotimo do i even wanna know what "WindowsBestFit" is all about %)
Zoffix :) 23:46
timotimo there's also a mapping for Atari ST and TT character maps
this code map interestingly contains the big integral that's made up of three code points (top, middle, bottom)
# This table contains the data the Unicode Consortium has on how 23:47
# IBM PC memory-mapped video graphics map into Unicode.
for some reason all of this makes me a little bit giddy 23:48
Zoffix wow, it's 8PM already. Kinda feel bad for these plumbers, since I still hear them banging on pipes. Still no water :|
timotimo :(
Zoffix Also, TIP: if building manager posts notice that hot water will be off. Fill some jugs with water, because they might turn off all of it. I'm like a dumbass with half a kettle here all day. 23:51
timotimo i'll go to bed now 23:57
i hope your pipe situation gets better soon :( 23:58
Zoffix night

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!