Parrot 4.5.0 "Buff-faced Pygmy Parrot" | parrot.org/ | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 29 June 2012.
whiteknight pmichaud: I suspect TOTAL_PMCs is the total that have been allocated. 01:25
ACTIVE_PMCS is probably the number of PMCs which are currently alive (or alive-ish)
probably 01:26
I have to read through the code
dalek p: bec0eba | pmichaud++ | src/HLL/Compiler.pm:
Restore a basic dumping capability from Parrot's Data::Dumper.
 01:41
whiteknight pmichaud: I'll take a look at the code tomorrow to verify. 02:10
02:26 wagle joined 02:28 wagle joined
pmichaud aloha: msg whiteknight (ACTIVE_PMCS) it would be really helpful to have that verified (with the gms gc). The numbers I'm getting back from some NQP compiles look really.... bizarre 03:16
aloha pmichaud: OK. I'll deliver the message.
pmichaud thanks, aloha
04:05 alvis_ joined 04:16 wagle joined 04:28 wagle joined 05:00 birdwindupbird joined
dalek p: 2bd8dcb | pmichaud++ | src/HLL/Compiler.pm:
HLL::Compiler --stagestats=3 (or greater) now introduces prompts after each compilation stage.

check for system resource usage.		 05:06
05:32 wagle joined 06:23 zby_home joined 08:59 lucian joined 11:54 whiteknight joined 11:57 Liz joined
whiteknight good morning, #parrot 12:06
12:08 PacoAir joined
whiteknight msg pmichaud: src/gc/gc_gms.c:1339. It appears to be a count of all PMCS which are not in the free list. This includes PMCs currently alive, and PMCs in older generations which are dead but no yet swept 12:15
aloha OK. I'll deliver the message.
12:33 lucian joined
dalek sella: c5fc925 | Whiteknight++ | / (2 files):
[String] Fix handling of {{ in FormatPrinter. Add a test
 13:09
sella: a5c3335 | Whiteknight++ | / (2 files):
[String] Update FormatPrinter.format_obj to take field-level formats as well, for more fine-grained control
sella: 7bb8a07 | Whiteknight++ | src/unstable/xml/Visitor.winxed:
[Xml] Add some function-level docs for Xml.Visitor
sella: 90933d1 | Whiteknight++ | src/string/FormatPrinter.winxed:
[String] Add some docs for FormatPrinter
13:10 Psyche^ joined
dalek sella: a38f3bc | Whiteknight++ | / (3 files):
[String] Rename FormatPrinter methods for consistency with other StringFormatters.
 13:20
14:41 kid51 joined 15:32 fperrad joined
dalek sella: 21a4351 | Whiteknight++ | src/unstable/ (3 files):
[Xml] Add Tag.add_children and Tag.add_children_from_hash methods, to help with constructing XML documents from native data types
 16:16
sella: 16949a7 | Whiteknight++ | src/unstable/xml/Document.winxed:
[Xml] Autogenerate an XML header if we don't have one already.
sella: 63d4926 | Whiteknight++ | src/ (2 files):
[Feed] Add a set_feed_details method to the Atom feed writer. A few other small fixes
16:29 lucian joined 17:13 jashwanth joined 17:57 zby_home joined 18:27 birdwindupbird joined
dalek : c526c01 | kjs++ | src/ (7 files):
improve type handling in parser and sem checker.
 18:34
: d09d3d2 | kjs++ | src/ (4 files):
cleanups.
: 5bd1885 | kjs++ | src/ (3 files):
don't store ints < 256*255 && >= 0 in consts segment as they're loaded with set_imm.
: eddba64 | kjs++ | src/gencode.c:
add code for preloading constants 0 and 1.
18:44 kjs joined 18:51 kjs joined 18:54 kjs_ joined 18:55 birdwindupbird joined 19:32 contingencyplan joined 19:35 Liz joined 19:47 kjs joined
dalek kudo/nom: 1edbd37 | moritz++ | / (2 files):
anonymous subset types with "subset :: where ...;"
 19:54
20:06 perlite joined
dalek sella: 07f8594 | Whiteknight++ | src/unstable/xml/Tag.winxed:
[Xml] Fix Tag.to_xml to be more compact when the tag has zero children or exactly one Text child. Add a Tag.last_child method to mirror the .first_child method
 20:16
sella: ecebcab | Whiteknight++ | src/unstable/xml/tag/Collection.winxed:
[Xml] Add methods to Collection for accessing items
sella: e0d1fb8 | Whiteknight++ | src/unstable/feed/Writer.winxed:
[Feed] Make sure to add some rel= attributes to link tags in Atom feeds, when necessary
20:27 timo left 20:29 lucian joined 20:43 kjs joined 21:02 birdwindupbird joined 23:14 davidfetter joined
pmichaud There's been some recent discussion on perl6-compiler and other places about Rakudo's build memory requirements, so I'm trying to understand Parrot's memory allocation a bit better. (more) 23:28
I have a small test program that allocates and releases PMCs and I get some odd (to me) memory statistics.... gist.github.com/3030004
can someone confirm that this is expected behavior for Parrot? 23:29
oddly, it appears that no gc runs took place in the execution of that program. If I increase the number of RPAs created to 10 million (from 1 million), I get some GC runs: gist.github.com/3030034 23:34
but by this point Parrot has allocated over 192MB of ram to hold one active PMC.
(if this belongs in a different forum such as an issue ticket or a mailing list, let me know.) 23:35
Coke pmichaud: trying running it under a ulimit? 23:36
since you can't tell the PVM to only use X Mb at runtime, it may never feel any pressure to collect.
pmichaud it still feels a bit excessive to me for some reason. 23:37
Coke (granted, it should do better than "never free anything" even without this hint.)
yah, seems crazy.... does it do any better run through the optimizer? 23:38
pmichaud ...optimizer?
whiteknight ...optimizer? 23:40
pmichaud what option would I give with ulimit to limit the amount of memory available?
whiteknight no such limit
there's a long-standing open ticket for that issue 23:41
it bounces between "can't fix" and "maybe later"
pmichaud: I don't know how the memory threshold is calculated, but I think it's a percentage of available memory 23:42
so it's entirely possible that a million entries might not trigger GC on your box
dukeleto pmichaud: i just commented on your gist
pmichaud: you might try using a PIR function instead of goto+label to see if the stats change dramatically 23:43
whiteknight no, that measures the CallContexts allocated
it would turn the number of pmcs from one million to two million
dukeleto pmichaud: my hunch is that you are creating many PMCs in the same scope and that may be a situation where our default gc (gms2) does poorly
pmichaud: i understand that you are trying to emulate how the rakudo build works and I also agree that our gc should do better than it is currently performing 23:44
pmichaud dukeleto: I'm trying to get a sense of how much memory the rakudo build is actually taking, and what is eating up all of the memory
so really I'm looking for ways to measure that. 23:45
whiteknight I don't think the scope has anything to do with it
pmichaud it looks like the interpinfo stats don't really help me much there, though
whiteknight the GC isn't limited by context
dukeleto whiteknight: --gc-nursery-size is my default, 2
pmichaud: i would also play around with --gc-nursery-size . The first 2 generations of allocated stuff get collected much less frequently than everythign else, iirc 23:46
pmichaud: you also may want to use --gc-dynamic-threshold during the Rakudo build 23:47
whiteknight pmichaud: Is your concern trying to get an accurate (and sensible) reading of memory usage, or to decrease it?
pmichaud how the heck do I do that?!
whiteknight: both, but in order to do the second I think I have to have the first.
if there's a way I can force gc runs, that might help. 23:48
whiteknight --gc-dynamic-threshold is the threshold at which a GC run is triggered, as a percentage of total available memory, I think
dukeleto pmichaud: you can force gc runs with a relative ( X mb ) or dynamic (X% of ram) flag, but tuning them is very machine-specific
pmichaud when we're doing the expensive part of the Rakudo build, it's Rakudo itself that is running.
it's not a Parrot command.
whiteknight I can't remember the details, I can look it up if you want
dukeleto pmichaud: you would then have to build a knob in Rakudo such that you can set those values from PIR/C 23:49
pmichaud so how do I set those from PIR/C?
dukeleto pmichaud: you can definitely set them from C, I am not sure if there is a PIR api for them. 23:50
pmichaud is there a C api?
dukeleto pmichaud: if you poke into how the --gc-* command line flags are implemented, they just set various flags in the C API, iirc
pmichaud or would this be an instance of "rakudo poking into parrot's guts"? ;-)
dukeleto pmichaud: that code was written by nwellnhof++, i believe
whiteknight I don't think pbc_to_exe generates a binary that can take that option 23:51
we would have to add it
pmichaud (force gc runs) I meant "Is there a way to force gc runs from PIR" 23:52
whiteknight yes, there's a sweep opcode
pmichaud ...checking
whiteknight sweep 1
pmichaud okay, that seems to do it.
whiteknight that opcode *should* trigger GC unconditionally
pmichaud with sweep 1, I get (one million RPAs): 23:53
gc_mark_runs=1
whiteknight it's not used often, and testing it is a bear, so I wouldn't be surprised to hear it's gone bitrotten
pmichaud gc_collect_runs=0
total_pmcs=1004190
active_pmcs=1588
total_mem_alloc=64913408
total_mem_used=285875
whiteknight okay, yes. That seems better
pmichaud 1588 is about right for active PMCs, I think.
whiteknight yeah. Our "base" level is far too high
every built-in class is a PMC, the interp, globals, every METHOD, etc 23:54
so there's a minimum which is depressingly high
dukeleto pmichaud: this area of code may help you github.com/parrot/parrot/blob/mast...ain.c#L473 23:56
pmichaud surely initargs is long gone by the time rakudo is running, though, yes? 23:57
whiteknight pbc_to_exe sets up a simple version of it with default values to start the interp
pmichaud or is initargs the structure that gms2 looks at for its threshhold information?
dukeleto pmichaud: the latter, iirc
whiteknight That structure is used to initialize the interp and gc_private structs, which is what the GC looks at at runtime
pmichaud yeah; that sounds to me like we'd be poking around in parrot's internals a bit too much. We're really trying to avoid that where possible -- I don't think we actually do much poking these days. 23:58
anyway, I can play with it a bit if need be.
whiteknight pmichaud: if you have any specific feature requests don't hesitate to ask 23:59

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!