Parrot 4.5.0 "Buff-faced Pygmy Parrot" | parrot.org/ | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 29 June 2012.
pmichaud ooc, would there be a way to use the mark algorithm to find all of the PMCs referenced from a given PMC? 00:00
whiteknight pmichaud: I don't *think* so right now. But I suspect such a thing could be added 00:01
Actually.... Maybe not
I'll have to look at it 00:02
dukeleto pmichaud: i would also suggest sending an email to parrot-dev and asking for input from bacek++ and nwellnhof++, who know the gc internals best 00:04
pmichaud: they don't seem to be on IRC very often, but they usually chime in on parrot-dev when they are needed 00:05
pmichaud dukeleto: will do. 00:06
'sweep 1' helps a bit, though; it helps to distinguish what PMCs are truly "alive" and which are just hanging around waiting to be collected
whiteknight I wonder if the profiling core could be augmented to show allocations and deallocations through valgrind 00:12
dukeleto whiteknight: sounds useful 00:17
dalek sella: 7a79016 | Whiteknight++ | src/date/Date (2 files):
[Date] Add some new ways to get shorter weekday and month names, for formatting
 00:48
sella: 3de1543 | Whiteknight++ | src/unstable/feed/ (2 files):
[Feed] Several fixes so we can write a basic RSS feed
sella: cbc78ca | Whiteknight++ | src/unstable/feed/Writer.winxed:
[Feed] Add version='2.0' to the <rss> tag.
Coke I literally meant 'ulimit', not an option to parrot to limit memory. 00:49
the problem with calling sweep is that, if it's anything like java, you end up forcing an expensive option more often than you need to. 00:51
00:51 kid51 joined
sorear pmichaud: you want ulimit -v KILOBYTES 00:51
whiteknight Coke: Yes, you don't want to force GC too early in terms of performance, but it's necessary in terms of analytics 00:55
and if it's an option you can turn on later, that's fine
02:06 aloha joined 02:39 bacek_at_work joined, brambles joined 02:40 brambles_ joined 04:07 davidfetter joined 04:42 japhb joined
dalek kudo/nom: 16db643 | pmichaud++ | src/core/Str.pm:
Add fix for Str.perl to be more robust when ICU isn't present (RT #113886).
 06:02
07:02 fperrad joined 07:11 brrt joined 07:47 Khisanth joined 08:26 lucian joined 08:27 Liz joined 08:28 preflex joined 10:53 Patterner joined 11:33 schm00ster joined 11:53 kid51 joined 13:01 PacoAir joined 13:08 whiteknight joined
whiteknight good morning, #parrot 13:09
brrt good afternoon 13:11
oh, whiteknight, i have some evidence that the embedding api doesn't properly catch all errors 13:30
i.e.
it sometimes causes my web server to exit()
which is a /huge/ pain
and wrong, and evil, etc
13:39 bluescreen joined
dalek p: f16f2c5 | jnthn++ | src/how/NQPModuleHOW.pm:
Stop nqp::can(EXPORTHOW, 'foo') from exploding by adding some missing bits to NQPModuleHOW.
 13:49
whiteknight brrt: Blah, sounds wonderful. 13:59
brrt: Do you have an example?
brrt: There are some errors in Parrot's core, like panics and core dumps and those things that do force an exit() call. 14:00
I've tried to minimize those kinds of things
brrt no, a regular exception
i'm going to try and recreate it outside of apache
whiteknight okay, that's extremely worrisome
brrt but it has been hard
it is 14:01
whiteknight Okay, pass any information to me as soon as you get it. We don't want that kind of creepy behavior happening
github.com/parrot/parrot/blob/mast...vate.h#L17 14:02
For your records, the embed API sets up a jmpbuf in the interp. When we throw an exception, if there are no handlers, the interp jumps to that jmpbuf 14:03
brrt oh.. wait
thats interesting 14:04
what happens if the jmpbuf cannot be set?
whiteknight It may be possible that the interp->api_jmp_buf field gets corrupted somewhere, or maybe if we throw an exception from a child interp that field might not have been initialized
brrt: gnashing of teeth, weeping of women, the end of days
brrt hahaha
whiteknight If the jmpbuf is null, I think we give up and exit 14:05
brrt lets see if i can reproduce this with the minimal amount of effort
exit with.. which response code?
whiteknight whichever we've passed to the routine
brrt well, the program exits with one
whiteknight github.com/parrot/parrot/blob/mast...exit.c#L70
Parrot_x_jump_out is what we call to exit the interp and fly back to that jmpbuf 14:06
brrt i trust that function
.. .i assume that jmpbuf becomes null somewhere
whiteknight So whatever status value is passed to that is what's used. That value is extracted from the exception object, and I think it defaults to 1
do a search for "Parrot_x_jump_out(interp, 1)" 14:07
actually, that probably wont find anything
Can you attach a debugger to apache and set breakpoints? 14:08
brrt i can very easily attach a debugger to apache, 14:09
that is called 'make debug' ;-) 14:10
but breakpoints, i do not know how
oh, break 14:11
whiteknight "break Parrot_x_jump_out" 14:13
Just a warning, gdb doesn't seem to handle longjmp very well. If you try to step through that, you'll get very confused about what's happening next
but breaking until that point and checking the backtrace will yeild lots of good information
brrt ok, maybe i can see if you can get the same results 14:14
do you have a recent checkout?
or at least, my last?
hasn't updated since friday iirc 14:15
whiteknight I'm probably a few days behind that
I'll update and test tonight when I get home, if you want
brrt yes, i'd like that
whiteknight sure thing
brrt ... hmm, it doesn't really seem to work 14:16
(breakpointing)
thats annoying
it seems to have not defined the functions on start
whiteknight yeah, mod_parrot is probably loaded dynamically 14:17
double-check your spelling and just continue
brrt apache is quite resilient against debugging as far as i can see 14:18
but, i can probably recreate the enviroment
.. i'm going to try it out at home 14:20
i can get the 'crash' to work easily, but i can't get any breakpoints
whiteknight okay, that's wonderful 14:25
brrt maybe exit
maybe exit() can be breakpointed
who knows
whiteknight maybe 14:26
brrt nope, can't be set, not here at least 14:27
whiteknight blah
14:56 whiteknight_ joined 14:59 alester joined
pmichaud github.com/parrot/parrot/issues/795 14:59
atrodo pmichaud: that seems useful... 15:03
whiteknight pmichaud: I 15:07
'll look at it tonight
15:09 dmalcolm joined
whiteknight I *suspect* the threshold calculation to determine the amount of memory to use before running a GC sweep doesn't take ulimit into account 15:13
so it thinks it has more room to grow, attempts an allocation, the allocation fails and it panics 15:14
pmichaud the corollary to the above report is that any Parrot program of any size will ultimately use at least 2% of the available memory, whether it actually needs that much memory or not. On my system, 450000 sub calls is sufficient to cause the process to allocate 180MB of memory
whiteknight right, that's the dynamic threshold at work
pmichaud whiteknight: I would think that if an allocation fails, though, that it would at least try a GC sweep before panicking.
whiteknight pmichaud: yes. It should 15:15
and, after I'm done with it, it will
pmichaud ooc, is it really a "dynamic" threshold if it's not responding to actual memory constraints? 15:16
seems to me that the threshold is more of an "initial" threshold than a dynamic one. 15:17
I guess it just depends on what one considers "dynamic". :-)
afk for a while 15:18
15:31 contingencyplan joined
Coke pmichaud++, nice ticket. 15:33
15:37 jashwanth joined
whiteknight the "dynamic" part of the threshold has to do with the total amount of memory. I don't know if parrot can query the current ulimit 16:02
I have to look at it
dalek p/toqast: e5cccaa | jnthn++ | src/QAST/Compiler.nqp:
Provide a way to handle loadlibs.
 16:15
kudo/toqast: a9770e0 | jnthn++ | src/QPerl6/Actions.pm:
Translate GLOBAL installation (still something vm-specific for now).
 16:19
kudo/toqast: 6e0b080 | jnthn++ | src/QPerl6/World.pm:
Some QAST translation of setting loading; still will need some generalizing by VM.
kudo/toqast: f62e3e3 | jnthn++ | src/QPerl6/World.pm:
Update loadlibs handling to QAST::VM approach.
p/toqast: 2e6ad4a | jnthn++ | src/HLL/Compiler.pm:
Make HLL::Compiler more aware of QAST.
 16:26
16:43 Liz joined
dalek p/toqast: a39b82e | jnthn++ | src/QAST/Compiler.nqp:
Improve error reporting when we find a PAST node inside a QAST tree.
 16:45
16:46 Liz joined 16:54 tuxit joined 16:55 Liz joined 16:58 LizM joined
Coke how to watch a ticket in github? 17:08
whiteknight no idea. 17:22
17:24 lucian joined 17:49 rich joined 17:57 dukeleto joined
dukeleto jashwanth: i am around if you need some help, but I will be on a phone call in a few minutes 17:58
17:58 woolfy joined 19:01 brrt joined
brrt ping whiteknight 19:01
whiteknight pong
19:02 plobsing joined
brrt so, at home, gdb works 19:02
and whats more 19:03
api_jmp_buf is very much zero
whiteknight okay, do you have the backtrace?
brrt ehm, i guess 19:04
whiteknight if you can gist that, I'll take a look at it
brrt will do 19:05
.... it doesn't crash now 19:10
oh, this is very interesting 19:11
i have a separate case which does not crash 19:12
very interesting indeed 19:13
gist.github.com/3035045 19:15
is the gist
... i just can't get it done outside of apache 19:18
19:18 rich left
brrt extermely similar code in all respects, doesn't crash 19:18
wait
nope, doesn't happen 19:19
19:21 brrt1 joined
whiteknight hmm 19:24
That's extremely weird
19:25 brrt1 joined
whiteknight it definitely should not be null there 19:26
19:27 brrt2 joined
brrt2 my connection is a bit flaky today 19:27
19:28 brrt joined
whiteknight I see that 19:28
brrt :-)
on my mobile phone now
much more reliable
whiteknight if it only happens in apache, I wonder if that means apache does something weird with setjmp 19:30
actually, that doesn't make any sense either
brrt ioh, but hilariously, it doesn't in all cases
whiteknight awesome 19:31
I'll have to step through it at home, to try and figure out when that field becomes null
brrt ok, its easy to reach 19:32
19:32 birdwindupbird joined
brrt basically, edit echo.winxed to throw an exception at the last line 19:32
whiteknight ok
Can you write me up all the details as an email? 19:33
brrt will do
whiteknight that way I wont need to ask a bunch of questions while you're sleeping
brrt ... could it have something to do with dlfunc? 19:35
no, not likely
.. or maybe 19:36
whiteknight you're very decisive
19:39 brrt joined
brrt i am, am i not? :-) 19:39
19:39 dukeleto joined 19:49 dukeleto joined
brrt it does not depend on nci 19:56
full report per e-mail
well, i have an answer, i think 20:02
it depends on the case where an NCI-called function (so, from within the interpreter) calls Parrot_api_* functions 20:03
in my case
to make a hash
which causes the interp->api_jmp_buf to be zeroed
which does not meet my personal definition of either awesome or correct 20:05
dukeleto brrt: indeed. we call that LTA around here: "Less Than Awesome"
brrt :-) 20:06
thats a good term for this 20:07
whiteknight oh yeah, Parrot_api_* are not reentrant 20:09
you can't call Parrot_api_* from NCI
brrt .. o 20:12
well
that is LTA
or, challenging
however you want to put it
or, a bit of a setback 20:13
hmm... i'll have to think about this 20:14
whiteknight you can use NCI to call the functions that the api routines call 20:16
What are you trying to do with that? 20:17
brrt i create a hash
basically, i create a hash, then iterate over the apache-specific table structure, and fill the hash with that data 20:18
whiteknight okay
brrt that hash is then returned to the 'loader' 20:19
whiteknight look in src/hash.c for routines to work with hashes
brrt i.e., the interpreter
whiteknight or the hash PMC
brrt i guess i should try that again 20:20
wasn't the greatest of successes tbh
whiteknight ok
brrt lots of header loading issues
but yeah, i see where the api jump buffer is zeroed
its in EMBED_API_CALLIN 20:21
whiteknight yeah
brrt could conceivably use a variable to hold the old jump buffer? 20:22
or is that a bad idea for reasons i do not know?
whiteknight We can probably set it up to use a linked list to make it reentrant
I'll look at it
brrt the other strategy is (still) extening parrot
creating a pmc to 'wrap' the apache data types 20:23
or use StructView
well, thanks anyway :-) this cleared up a lot
whiteknight no problem
I'm wrapping up now. I'll look at some stuff later tonight 20:24
brrt bye 20:25
brrt is leaving as well &
20:25 brrt left 20:44 brambles joined 21:03 woolfy joined 21:24 LizM joined 21:25 LizM joined, woolfy joined 21:31 woolfy left 21:46 LizM joined 21:47 woolfy joined
dalek rrot/chromatic/early_pmc_gc: 2cd0e7e | chromatic++ | / (2 files):
Added early GC PMC reclamation to GC GMS.

This makes the "sweep 0" opcode reclaim all PMCs in the youngest generation which have their early GC flag set. Set this with the "needs_destroy" opcode.
If you don't know what you're doing, you can cause great harm. If you do know what you're doing, you can force a cheap GC run every few thousand PMC allocations to reclaim PMCs you know won't be used ever again without running a full mark and sweep or waiting to bump up against a memory limit.
This comes with the appropriate warning: YOU are performing manual escape analysis. If you get that wrong, you'd better have a spectacular test suite to demonstrate that you have it wrong.		 22:04
rrot/chromatic/early_pmc_gc: 7ec2caf | chromatic++ | src/gc/gc_gms.c:
Ran headerizer on new lazy GC code.
rrot/chromatic/early_pmc_gc: 8047502 | chromatic++ | src/gc/gc_gms.c:
Added documentation to new GC GMS functions.
rrot/chromatic/early_pmc_gc: 1742672 | chromatic++ | src/gc/gc_gms.c:
Added headerization.
22:17 whiteknight joined
whiteknight good afternoon, #parrot 22:28
22:37 plobsing joined
pmichaud 17:08 <Coke> how to watch a ticket in github? 22:39
At the bottom of the ticket page, there's an obscure link that says "Notifications for new comments on this issue are {on|off}". Make sure it's set to "on". :-)
I'm guessing that will enable email notifications for that issue for you. 22:40
23:02 kid51 joined 23:16 woolfy left 23:36 lucian joined
whiteknight msg moritz: I've tested the parrot-iocleanup1 branch against my most recent changes. Looks like the encoding-related issues are gone, but we've still got the numchars issues. How do you want that fixed? 23:37
aloha OK. I'll deliver the message.

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!