6.2.11 released! | pugs.blogs.com | pugscode.org | pugs.kwiki.org | paste: sial.org/pbot/perl6
Set by wolverian on 12 February 2006.
Alias_ ah, right 00:00
beppu kinda different from typical unix, but it works for perl.
Alias_ right
beppu ...so you solved these problems fairly recently, then.
Alias_ When audreyt stayed with me for the week in December 00:01
She knew how, and I like making things simple
File::HomeDir and File::UserConfig are as a result of a grant to do a desktop application 00:02
beppu I'll try it out for the app I'm writing.
Alias_ let me know how it goes
Sorry I don't have system etc and htdocs, or web cache, or data cache yet :)
But I have some ideas
beppu Sometimes (in my unix-centric world) I forget about other operating systems... but this scheme seems like it'll work portably.
Alias_ The problem is that if you just go implement it, you have "where Alias things cache should go", not necesarily where it actually should go 00:03
So it's high risk to just start coding without lots of research and confirmation
s/things/thinks/
A Perl /var/cache might be ok though 00:04
oh, also we have a standard /tmp
:)
So that's tmp, bin, lib, share, and user
svnbot6 r9056 | audreyt++ | * DrIFT.RuleYAML - add diagnostics messages when loading 10:17
r9056 | audreyt++ | of YAML structures failed.
r9057 | audreyt++ | * re-drift the four source files. 10:20
r9057 | audreyt++ | * (previous commit also unbroke "make profiled".)
azuroth I did some of my C++ homework in haskell. teacher didn't like that :-( 11:30
Kattana and this surprises you? 11:33
svnbot6 r9058 | gaal++ | * Fixed remaining kinks in precompiled Prelude loading. 11:56
r9058 | gaal++ | Ladies and gentlemen, it works! and it's fast!
gaal time ./pugs -CParse-YAML -e 'our $*str = "hello"' > h.yml 12:02
0.040u 0.110s 0:00.14 107.1% 0+0k 0+0io 0pf+0w
time ./pugs -e ' Pugs::Internals::eval_p6y("h.yml"); say $*str'
hello
0.470u 1.120s 0:01.59 100.0% 0+0k 0+0io 0pf+0w
and, for simple one-liners:
time ./pugs -e 'say "hello, fast world!"' 12:03
hello, fast world!
0.460u 1.070s 0:01.58 96.8% 0+0k 0+0io 0pf+0w
... and this is on a mostly unoptimized pugs :-)
audreyt gaal++ gaal++ gaal++ 12:13
clkao (!) 12:15
Juerd gaal: Hey, that's weird, in your character set, numbers turn out much lower! :) 12:16
gaal++
Any figures yet on how much this saves a smoke?
audreyt well, it saves compilation time mostly 12:19
the runtime doesn't get faster Ithink
gaal Juerd: not yet. but one significant benefit is that people with low RAM can still build a useful pugs 12:20
that doesn't take 35 seconds to say hello world.
the new system makes loadtime a little slower, in fact, but as audrey points out, we can now cleanly also precomplile Test.pm 12:21
so it should pay off nicely.
Juerd audreyt: Oh, I misunderstood the figures then. I thought startup time was decreased.
gaal also also, audrey has been profiling and speeding up -CPugs 12:22
Juerd Ah
gaal Juerd: one thing that's massively easier now is, uh, separate compilation of arbitrary code
./pugs -CParse-YAML -e your-code > a-dump.yml 12:23
and load it with a new prim
and we've been thinking of design for module bytecode
there are a few very good ideas to steal from python, for example, precompile any module you encounter, if you have write permissions to the directory it is in 12:25
they also have a very good design for the bytecode format we can plag^H^H^H^Hstudy and learn from 12:28
so this means that when you install a module from CPAN, pugs can not only throw .p6 files in your install site, but also precompiled .p6c (or whatever we name them) 12:29
audreyt maybe .p6c for precompiled unoptimised, .p6o for precompiled optimised (closed-finalized with "use optimize") 12:37
gaal okay, some more data points. on a modest machine (p3 1.2GHz, 256MB): 12:40
clean build, unoptimized: about 13 minutes
(this includes the JS prelude precompilation, which was optional, and took about 2 min)
./pugs -e 'say "hello world"' took 8 seconds to run 12:41
which is eminently usable
svnbot6 r9059 | audreyt++ | * -CPugs: Massively speeded up compilation speed and loading speed.
gaal much better than the 35-odd seconds that it used to take.
pdcawley Cool!
What's that done for the speed of running the full test suite? 12:42
gaal and, of course, memory consumption was reasonable
pdcawley: we need to benchmark. at first instance, it will slow it down
pdcawley ? what? More stuff happens at runtime now or something? 12:43
gaal as the backlog explains, loading is now done from yaml, which is fast, but at runtime is a little slower than the old -CParse mechanism
Juerd A psychological trick to pull is to include this precompilation in the "make" phase, which makes the "make test" phase benchmarkable between the first and second time.
gaal however, we can now easily and cheaply precompile Test.pm
lypanov pugs is slow anyways, devel speed is way more important :)
Juerd And make it look like it's fast ;)
gaal so it will pay off
additionally, audrey has just checked in a speedup to -CPugs :) 12:44
pdcawley Right, 'cos you're no longer having to compile Test.pm for every test.
gaal yes, so that's like $dot_t times the parse hit for Test.pm
pdcawley $dot_t ?
gaal find t ext -type f | wc -l :-) 12:45
pdcawley Should make chromatic a little less unhappy...
gaal so, to summarize the prehackathon:
even if we revert to the old precompilation strategy of -CParse, we will have gained two things: 12:46
1. knowledge about separate compilation with YAML 12:47
Juerd How fast is pugs at skipping POD?
gaal 2. -CPugs itself is now faster, thanks for our (audrey's) profiling work
Juerd And if not fast, then: does pugs support __END__ yet?
integral are there lots of other places in pugs that could benefit from FPS?
gaal Juerd: benchmarks welcome :)
Juerd gaal: I understand. Will have to compile first 12:48
pdcawley FPS?
integral Like, should VStr be changed to a FPS?
pdcawley: Data.FastPackedString
pdcawley Haskell datatype?
Juerd 13:59 < gaal> 2. -CPugs itself is now faster, thanks for our (audrey's) profiling work
gaal integral: the parser could profoundly benefit from it, as will pugs in general, but that would probably mean switching away from Parsec, as it's just too closely coupled to haskell SlowStrings.
Juerd Any figure? (%?)
integral gaal: that's because Parsec will work with parsing any list type? 12:49
audreyt Juerd: let me finish the FPSization
and I'll give you bench numbers
Juerd audreyt: Oh, didn't know you were already even more steps ahead. Don't let me distract you :)
gaal integral: yes
Juerd (Should have guessed, or at least assumed :P) 12:50
gaal Juerd: of course, we're going off to meet Larry in half an hour, so there are portions of the world to take over by then ;)
Juerd Oh, interesting :) 12:51
This is interesting too:
Writing Makefile for WTemplate 12:52
Writing Makefile for Perl6::Pugs
Segmentation fault
But it's DProf that causes this
So no worries
bsb Re: borrowing from python, it seems like their header could've been better planned 12:53
mail.python.org/pipermail/python-de...04271.html
gaal ooh, we were looking at the header and were thinking it was pretty good. /me looks 12:54
bsb An offset to the code could have allowed future room to grow
integral it needs to be a binary header? It can't just look like a normal perl file but with a 'use MagicPseudoSourceFilter 0.45'?
bsb If these file are to be binary then www.w3.org/TR/PNG-Rationale.html#R....-signature is a good reference 12:55
gaal cool (the \r\n trick is really neat) 12:56
bsb for sure
gaal always says #!/usr/bin/perl -w on the hashbang as a newlinality shield 12:57
bsb eh? 12:58
gaal separate but similar issue:
perl5 is clever enough not to care about newline types
but the if the kernel doesn't find an interpreter called "perl\r", it barfs
with the infuriatingly misleading error message about file not found 12:59
bsb I once had trouble with -F on the #! line, adding -w fixed it :)
integral hmm, empty almost needs a typeclass
oh, mempty, duh 13:00
gaal integral: what's mempty?
integral unit of the Monoid class
gaal integral: that's still audrey stuff as far as my brain's concerned :) 13:02
gaal 's brain has made many protests since Tuesday, not all of them successful 13:03
integral hmm, is drift slow on Pugs.AST.Internals for everyone? 13:27
Juerd 14:09 * gaal always says #!/usr/bin/perl -w on the hashbang as a newlinality shield 16:08
Ah, smart!
Thanks for the trick :)
Re byte headers: After the magic bit, I think things are better off as a hash-like structure, rather than a positional list of metadata, for scalability 16:11
And maybe it'd be nice even to store the code length as well as the bytecode data, so we can have multiple files encoded in a single file, or interesting stuff like DATA sections 16:14
integral maybe we should just use something compatible with PBC's?
Juerd What be PBC?
Parrot BC?
integral parrot bytecode container, or something
Juerd I don't know the format
integral it's yet another executable file format with multiple sections and so on. 16:15
Juerd Is it neat and scalable?
And does it have enough offsets or size indicators to be parsed without too much pain? :P
integral Probably not, but afaik you can invent new section types for it (like Jonathan did for debugging)
hope so. Remember the parrot folks are having to use C to do their parsing after all
Juerd Yes, but parsing for one purpose may be easy while parsing for another may be hard. 16:17
If you're writing an analysis tool, and you need it to be fast, so you skip the common libraries and stuff, you don't want to have to parse everything to get to the end.
integral www.parrotcode.org/docs/parrotbyte.html # They actually have docs! yay! 16:18
Juerd Say you have N/a* chunks of data as a header, and then a body that's more interesting. Then N/(N/a*) is nicer, because it lets you skip the header chunks entirely. 16:18
Huh, Parrot with documentation? Are you sure it's the same Parrot? ;)
integral don't worry, I'm sure it'll turn out to be unimplemented, or incorrect :) 16:19
Juerd Hm, no magic? 16:20
integral 16 bytes in
Juerd But...
Juerd sighs
Okay.
Heh, magic is native even.
That's not a useful magic for file(1)-like utilities, and the many libraries that do the same kind of thing. 16:21
integral oh dear.
Juerd We're not going to fully depend on .foo extensions, I hope.
integral I thought the parrot designers had all been unixy enough to spot things like that :-/
Juerd It's nice to have them, and even to require them, but when I loose the extension, I still want to be able to tell what the hell to parse the file with :)
I hope this is an unrelated kind of magic, because a highly variable magic part is useless for the kind I'm talking about... 16:22
I say let's steal PNG's magic ;) 16:23
But slightly different
To avoid endless images of talking birds. :)
nothingmuch audreyt: ping 19:46
gaal: ping
gaal pong 20:43
nothingmuch: hello! :)
nothingmuch gaal: hola 20:47
svnbot6 r9060 | audreyt++ | * test for the bug where lexically-imported functions
r9060 | audreyt++ | will incorrectly get rebound when the containing function
r9060 | audreyt++ | gets reentered.
r9061 | audreyt++ | * the (massively undocumented) Pugs::Internals::eval_p6y($file) 20:50
r9061 | audreyt++ | function now incoporates global bindings in the source yaml file.
r9062 | audreyt++ | * DrIFT.RuleYAML: Introduces the "YAML_Pos" drift rule, 20:53
r9062 | audreyt++ | to save the resulting .yml file's size by discarding
r9062 | audreyt++ | label names from Haskell record types and treat them
r9062 | audreyt++ | as positionals.
r9063 | audreyt++ | * Data.Yaml.Syck: trivial code cleanup of s/toEnum . fromEnum/fromIntegral/ 20:56
nothingmuch goes to bed 20:57
svnbot6 r9064 | audreyt++ | * Pugs.Compile.Pugs: -CPugs now uses FastPackedString;
r9064 | audreyt++ | the generated code is also much more compact (but we are
r9064 | audreyt++ | not using it for Prelude anymore.)
nothingmuch see you all tomorrow at the post-pre-hackathon
audreyt see ya nothingmuch
svnbot6 r9065 | audreyt++ | * When prelude loading failed due to (trapped) errors, make 21:00
r9065 | audreyt++ | it clear that we are reloading it again from parsed source code.
svnbot6 r9066 | audreyt++ | * PreludePC.hs - actually load global bindings from Prelude. 21:36
r9066 | audreyt++ | there are some remaining problems -- e.g. sprintf() complains
r9066 | audreyt++ | about arity errors -- to be investigated tomorrow.
integral audreyt++ # pugs feels much faster now 21:45
svnbot6 r9067 | audreyt++ | * PreludePC - use modifyTVar to union the glob pad on the 21:52
r9067 | audreyt++ | correct order (newest first). Somehow &*eval is still
r9067 | audreyt++ | broken. :-/
audreyt integral: &*eval is somehow broken though. (as is &*sprintf, or any code binding with arity>1) 21:53
gaal and I may look into this tomorrow -- but it's midnight and I should sleep now
& 21:54
integral hmm, just discovered that FastPackedStrings are 8-bit only :-/ 23:27
Juerd integral: If you strictly encode the same, that shouldn't matter much 23:41
Fortunately, utf8 is octet aligned :) 23:42

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!