smoke string_gc_encapsulate branch | Parrot 2.7.0 "Australian King" Released! | parrot.org Log: irclog.perlgeek.de/parrot/today | Nopaste: nopaste.snit.ch:8001 | close 25 tickets; remove deprecated items (especially CodeString); record deprecations; polish for 2.8 release
Set by moderator on 16 September 2010.
Paul_the_Greek If we trust the sizes, then we ought to be able to define the casting macros and complain if we can't. 00:00
00:00 mikehh left
NotFound Paul_the_Greek: the non uppercase names at one point was the recommended way for extenders. 00:01
Paul_the_Greek Yes, they look official. 00:02
NotFound Then people started to worry about why some header files looked different and were not headerized, and all changed. 00:03
00:04 whiteknight left 00:15 mikehh_ is now known as mikehh
mikehh opbots, names 00:15
00:22 nwellnhof joined 00:23 Paul_the_Greek left
nwellnhof the stress strings performance "regression" was probably caused by r48585. it changed the GC threshold back to a more conservative value. 00:28
pmichaud git bisect says that rakudo's unicode failures started with r49018 00:29
nwellnhof pmichaud: then it's my fault :(
chromatic The loss of encoding information, or an off by one error? 00:30
nwellnhof what are the failures exactly?
pmichaud I don't know what the actual problem is.
nwellnhof: some utf8 strings aren't being managed properly.
here's an example:
gist.github.com/583439 # example failure with r49018 00:31
note the o+diaeresis at the end of the variable name
if that variable name is made longer, it works
nwellnhof i think i found the problem. 00:32
pmichaud so it sounds like a length calculation error somewhere
nwellnhof strings from concat get the wrong encoding.
fix is coming up... 00:33
00:34 theory left
nwellnhof pmichaud: since a week or so, i get an infinite loop in Rakudo's make spectest 00:34
somewhere in S03 00:35
pmichaud can you narrow it down a bit further than that?
are you running tests in parallel?
nwellnhof i have to retest, wait a little... 00:36
pmichaud btw, this comment:
+ /* PObj is a copy of a string that doesn't own the string buffer */
+ PObj_is_string_copy_FLAG = POBJ_FLAG(10),
worries me a lot.
nwellnhof why?
pmichaud does one string "own" a string buffer? Is there something to prevent the string buffer from being reclaimed when the owner disappears and the copy still exists? 00:37
nwellnhof yes, there's exactly one string that owns a buffer. but it's not tied into the GC. 00:38
pmichaud when does the buffer get reclaimed?
nwellnhof by the GC. but that has nothing to do with the new string flag. 00:39
pmichaud okay.
nwellnhof the new string "ownership" only means that one string is allowed to extend its content into preallocated space. 00:40
pmichaud (not "okay, I understand and agree" but "okay, I'll take your word for it")
00:41 ruoso joined
mikehh All tests PASS (pre/post-config, make corevm/make coretest, smoke (#66), fulltest) at r49073 - Ubuntu 10.10 beta {g++-4.5 with --optimize) 00:41
dalek rrot: r49074 | nwellnhof++ | trunk/src/string/api.c:
[str] Set correct encoding in Parrot_str_concat
 00:49
nwellnhof pmichaud: i get an infinite loop when i evaluate (10 R... 1, 3) 00:51
test 33 in t/spec/S03-metaops/reverse.t 00:53
bacek_at_work nwellnhof, "rm -rf t/spec" 00:56
Perl6 spectest was migrated to github.
You probably have stale copy from Pugs svn 00:57
00:57 kid51 joined
nwellnhof that's it, bacek++ 00:58
kid51 ~~
kid51 studies Parrot Foundation documents 00:59
01:03 eternaleye_ is now known as eternaleye 01:31 dngor left 01:33 dngor joined
pmichaud r49074 needs a test, perhaps. 01:41
nwellnhof i just finished Rakudo's spectest
chromatic A test case in Parrot's suite. What would trigger it? 01:42
nwellnhof t/spec/S05-mass/properties-derived.rakudo failed, but that always fails for me. seems to be related to the ICU library.
and the gray code test in t/spec/integration/99problems-41-to-50.rakudo failed. 01:43
# got: '000 001 011 010 111 110 100 101'
# expected: '000 001 011 010 110 111 101 100'
ah yeah, a test case for r49074 01:44
i write one 01:45
pmichaud the 41-to-50 failure is a known failure 01:53
I'll either fix or fudge it here in a bit.
r49074 looks good here so far (up to S32-* tests) 01:54
kid51 r 49073 make fulltest PASS on Linux/i386 01:55
chromatic 49074 should be a little faster on the spectests, too. Faster than 49000 anyhow. 01:56
02:20 Andy joined, kid51 left 02:24 davidfetter left
dalek rrot: r49075 | nwellnhof++ | trunk/t/op/string_cs.t:
[str] Add test for r49074
 02:30
02:35 janus left
dalek TT #1535 closed by cotto++: trac+git demo site 02:51
TT #1535: trac.parrot.org/parrot/ticket/1535
kudo: 2c8bb89 | pmichaud++ | build/PARROT_REVISION:
Bump PARROT_REVISION.
02:52 bluescreen left 03:32 contingencyplan left 03:38 theory joined, gottreu joined 03:40 jsut_ joined 03:45 jsut left 03:56 janus joined
dalek TT #1477 closed by plobsing++: [embed] expose non-vararg-based sub/method call-ins 03:59
TT #1477: trac.parrot.org/parrot/ticket/1477
plobsing roadmap? 04:03
purl rumour has it roadmap is to see parrot roadmap
plobsing parrot roadmap?
purl hmmm... parrot roadmap is icanhaz.com/parrotroadmap or trac.parrot.org/parrot/roadmap
plobsing where is that googledocs roadmap? 04:06
btw, the icanhaz.com link is 404 04:07
04:12 nwellnhof left 04:58 Khisanth left 05:01 Khisanth joined 05:04 jsut joined 05:08 jsut_ left 05:14 rurban_ joined 05:16 rurban left, rurban_ is now known as rurban 05:50 davidfetter joined 05:55 Andy left 06:05 plobsing left 06:13 gottreu left 06:16 chromatic left 06:34 uniejo joined 06:49 esskar left 06:52 esskar joined
pmichaud rakudo spectest times on r48933: 22m23s (real) 43m13 (user) 06:58
rakudo spectest times on r49074: 20m06s (real) 38m46 (user)
Excellent work, everyone.
cotto is becoming more convinced that stealing speed.pypy.org would be a really good idea, if possible 07:38
or speedcenter, as the software behind it is called
Tene pmichaud: rakudo.org/status hasn't been updated cince early July, looks like? 07:41
07:49 contingencyplan joined, fperrad joined 07:52 contingencyplan left 07:54 contingencyplan joined
mikehh All tests PASS (pre/post-config, make corevm/make coretest, smoke (#74), fulltest) at r49075 - Ubuntu 10.10 beta (gcc-4.5) 07:55
07:55 contingencyplan left 08:01 contingencyplan joined 08:04 contingencyplan left 08:08 contingencyplan joined 08:09 contingencyplan left 08:12 contingencyplan joined
dalek rrot: r49076 | gerd++ | trunk/docs/project/release_manager_guide.pod:
indicate to have run "make" with the old version number
 09:14
lscript: 86a4ba4 | fperrad++ | dynext/pmc/wmls (5 files):
logical VTABLES are gone, see trac.parrot.org/parrot/changeset/49012
 09:42
rrot: r49077 | fperrad++ | trunk/src/pmc (2 files):
[pmc] honors HLL type
 09:48
10:35 ruoso left 11:04 fperrad left 12:02 whiteknight joined 12:06 lucian joined, uniejo left 12:10 contingencyplan left 12:22 ruoso joined 12:27 bluescreen joined 12:32 uniejo joined
dalek TT #1432 closed by whiteknight++: Remove 2-argument math ops 12:43
TT #1432: trac.parrot.org/parrot/ticket/1432
rrot: r49078 | whiteknight++ | branches/stdhandle_meths:
Creating a branch to work on the issues raised in TT #264
 12:52
12:56 uniejo_ joined, uniejo left 12:58 patspam joined 13:14 rurban_ joined 13:16 rurban left, rurban_ is now known as rurban 13:18 Psyche^ joined, Patterner left, Psyche^ is now known as Patterner
dalek rrot: r49079 | whiteknight++ | branches/stdhandle_meths/src/pmc/parrotinterpreter.pmc:
delete the stdhandles method and add new stdin_handle, stdout_handle, stderr_handle methods
 13:26
rrot: r49080 | whiteknight++ | branches/stdhandle_meths/compilers/pct/src/PCT/HLLCompiler.pir:
update uses of stdhandle method in HLLCompiler.pir
rrot: r49081 | whiteknight++ | branches/stdhandle_meths/tools/dev/nci_thunk_gen.pir:
update uses in nci_thunk_gen.pir
p-rx: 8009d57 | pmichaud++ | build/PARROT_REVISION:
Bump PARROT_REVISION.
 13:38
p-rx: c9a989b | pmichaud++ | / (2 files):
Fix negated zero-width enumcharlist at end of string (fixes issue #9 on github). Discovered by moritz++.
p-rx: ccc2e3f | pmichaud++ | src/stage0/ (4 files):
Update bootstrap.
13:39 bluescreen left 13:41 uniejo_ left
NotFound whiteknight: ping 13:43
High frequency sonar
dalek rrot: r49082 | pmichaud++ | failed to fetch changeset:
[nqp-rx]: Update bootstrap with fix for negated charclass at end of string.
 13:44
13:45 davidfetter left
NotFound msg whiteknight See my last comment on TT #264 before investing more time and effort 13:46
purl Message for whiteknight stored.
aloha OK. I'll deliver the message.
dalek kudo: ca4a1d6 | pmichaud++ | build/PARROT_REVISION:
Bump PARROT_REVISION.
13:53 bluescreen joined
whiteknight NotFound: I'm almost done anyway. It wasn't much effort. 13:57
I would prefer methods on the interpreter object instead of the opcodes, so I'm going to argue for that. I can take the fight to the list if necessary
13:59 fperrad joined
atrodo If I do a search on trac for 264, ticket #264 is on the third page of three 14:00
NotFound whiteknight: please wait after the release. 14:11
14:14 davidfetter joined
dalek rrot: r49083 | fperrad++ | trunk/t/dynpmc/rational.t:
[t] refactor with skip_all
 14:17
14:51 plobsing joined 14:53 jan left
mikehh atrodo: you need to specify #264 in the trac search and it goes straight there 15:08
atrodo Didn't realize the # was significant... 15:09
mikehh me either - until someone suiggested it 15:10
15:12 nwellnhof joined
moritz typically tipes tt #264 on IRC, and follows the link in the IRC logs :-) 15:14
*types
15:24 Andy joined
dalek rrot: r49084 | fperrad++ | trunk (2 files):
[osutils] add rindex()
 15:25
rrot: r49085 | fperrad++ | trunk/runtime/parrot/library/distutils.pir:
[distutils] refactor mk_path_gen_dynpmc with rindex
15:38 nwellnhof left
dalek ast: c19ab2c | KodiB++ | S02-builtin_data_types/set.t:
[set.t] Marked some tests as relevant to RT #77760.
 15:40
kudo: da7c7f9 | KodiB++ | / (2 files):
Tweaked Set enough to let us pass set.t.
 15:42
TT #103 closed by whiteknight++: No self in PIR invoke VTABLE 15:50
TT #103: trac.parrot.org/parrot/ticket/103
mikehh pmichaud: ping 15:51
16:10 jan joined
dukeleto hola hola hola 16:22
cotto ~~~ 16:25
moritz rakudo: print ~('hola' xx 5)
p6eval rakudo ca4a1d: OUTPUTĀ«hola hola hola hola holaĀ»
pmichaud mikehh: pong 16:28
cotto nice to see some sparc smolder reports 16:29
16:29 lucian left
dukeleto cotto: indeed 16:30
whiteknight it's nice to see some passing sparc smolder reports 16:31
dukeleto touche!
davidfetter hands dukeleto a few acute accents 16:32
16:37 rurban left
mikehh pmichaud: I am getting a codetest failure with ext/nqp-rx/src/stage0/HLL-s0.pir - pod syntax - I can fix it but that doesn't propogate 16:44
pmichaud ext/nqp-rx is supposed to be excluded from codetest, I thought.
HLL-s0.pir is generated code.
mikehh it seems to have removed a =over 4 and a =back from the pod 16:46
pmichaud: I also thought we were going to remove stuff in ext/ from testing, but it seems to just take files from MANIFEST 16:47
pmichaud can you show a diff or context around the =over 4 and =back that are missing? 16:48
pmichaud runs 'make codetest' locally to seek enlightenment
nopaste "mikehh" at 192.168.1.3 pasted "svn diff to get ext/nqp-rx/src/stage0/HLL-s0.pir to pass codetest" (22 lines) at nopaste.snit.ch/23414 16:50
dukeleto mikehh: that is generated code 16:51
mikehh: it shouldn't have to pass codetest
16:52 chromatic joined
dukeleto mikehh: but, methinks it should have valid POD. 16:52
chromatic: greetings
mikehh dukeleto: I agree, but the pod_syntax test seems to take files from MANIFEST
dukeleto mikehh: i can understand checking generated files for valid POD, but not any other code tests. 16:53
mikehh doesn't seem to fail or be run through any other tests 16:54
in fact it doesn;t run through some of the others 16:55
dalek p-rx: df8c49d | pmichaud++ | src/HLL.pir:
Add some pod directives to help out Parrot's "make codetest" target.
chromatic aloha 16:56
I wonder of a very strict C or strict-ish C++ compiler would consider the PL/parrot "I haven't seen this function header!" warnings as errors. 16:57
s/of/if/
dalek TT #1755 closed by fperrad++: Include arbitrary .c and .h files in distutils dynpmc build 16:58
TT #1755: trac.parrot.org/parrot/ticket/1755
dukeleto chromatic: possibly. 17:00
chromatic: i am still working on getting you some more useful benchmark data 17:01
chromatic Thanks. 17:02
dalek rrot: r49086 | fperrad++ | trunk/runtime/parrot/library/distutils.pir:
[distutils] allows .c & .h in dynpmc item, see TT#1755
 17:06
whiteknight fperrad++ 17:07
that's going to make some of the PLA configure code much much nicer
pmichaud Idle thought (as I head off to lunch): Would there be some way to get Parrot to tell me how many references there are to a given PMC...? 17:08
(could be calculated)
whiteknight at the moment: no 17:09
17:09 nwellnhof joined
pmichaud it could help in finding large data structures that would otherwise be release-able 17:09
moritz would that be useful?
whiteknight something that might be able to be instrumented with the instrumentation PMCs
jnthn pmichaud: I guess if the gc could be co-opted it could work.
pmichaud: Since it's essentially the same kind of walking 17:10
pmichaud moritz: sure. For example, after we finish compilation, we don't need the parse tree or ast any longer. But right now there's not any way to know what might be holding on to it.
moritz do you know if the parse tree is GC'ed? 17:11
whiteknight the problem is that right now we don't have any where to hold the count of incoming references
pmichaud moritz: I suspect it isn't being GC'ed. 17:12
whiteknight finding a way to add that information without adding an integer per PMC header is the challenge
moritz if we have custom destroy vtables, you maybe could find out
pmichaud whiteknight: I was thinking an opcode
whiteknight: i.e., "count_refs $P0"
jnthn whiteknight: Maybe it should do something like the GC does, but not actually use the GC.
pmichaud which does a mark run, and keeps track of the number of times that $P0 was encountered
and returns that value
chromatic How about stuffing fprintf() in the source code and post-processing the results? 17:13
whiteknight I think they want it on every run, in production
not just as a benchmark
chromatic ... before anyone goes all crazy designing a framework for this sort of thing.
pmichaud I'm thinking as benchmark/debugging aid
jnthn whiteknight: It's more a debugging aid.
pmichaud not production
whiteknight oh, then I misunderstood
pmichaud or, perhaps even better -- a way to trace a mark run
nwellnhof some kind of general GC logging framework would be really useful 17:14
pmichaud yes
nwellnhof also for bug hunting
pmichaud it could be a dynop -- but something that runs through a mark phase and reports back the references/steps
that could be *really* useful.
anyway, time for lunch. 17:15
whiteknight Khairul's instrumentation PMCs could easily do a job like that 17:17
moritz are they merged to trunk?
whiteknight no, not yet.
chromatic Hang on, tracking mark isn't nearly that easy.
whiteknight When I suggested merging, I still thought that his GSoC project involved changes to trunk' 17:18
instead, all his logic is neatly encapsulated into PMCs
moritz that's nice 17:19
whiteknight it is extremely impressive
unfortunately, his project doesn't build at the moment 17:20
dukeleto whiteknight: what error do you get? 17:38
chromatic msg plobsing Some very careful inlining of code in Parrot_hash_thaw() could give Rakudo a ~8-10% startup performance improvement. 17:41
purl Message for plobsing stored.
aloha OK. I'll deliver the message.
dalek rrot: r49087 | nwellnhof++ | trunk/src/string/api.c:
[str] Missing checks for end of string in Parrot_str_unescape_string
 17:56
rrot: r49088 | nwellnhof++ | trunk (16 files):
[str] Remove unneeded encoding header files
chromatic Wow, 2.591%. 18:00
nopaste "chromatic" at 192.168.1.3 pasted "Refactoring parrot_hash_put improves hash thawing performance" (187 lines) at nopaste.snit.ch/23416 18:07
chromatic Needs documentation, but that's a sizable improvement of Rakudo startup.
dukeleto nice work! 18:08
chromatic It probably generalizes throughout hash.c.
whiteknight chromatic: that refactor nets 2.59%? 18:12
that's pretty impressive
18:13 contingencyplan joined
chromatic When you work *with* C's type system and not against it, the optimizer kicks in. 18:13
nwellnhof i must be missing something. where are the type related changes in your patch? 18:16
chromatic Mostly in Parrot_hash_thaw.
Avoiding the "Is this a STRING key?" checks pays off.
nwellnhof ah, i didn't look there. i thought it had to do with splitting code out of parrot_hash_put. 18:17
chromatic Step one and step two. 18:18
nwellnhof yes, i see.
chromatic I'll write docs for the new static functions, then commit in two steps. 18:19
Anybody seeing failures in t/spec/S32-num/rat.rakudo, by the way?
nwellnhof and it even cleans up code, IMO
couldn't we use parrot_hash_get_bucket_string in parrot_hash_get_bucket, too?
18:20 fperrad left
chromatic Probably, and it probably is an improvement there too. 18:21
nwellnhof btw, parrot_hash_get_bucket_string *can* return NULL 18:30
chromatic Right.
dukeleto chromatic: what failures are you seetin in rat.rakudo ? 18:32
chromatic: are you running it through fudgeall ? 18:33
chromatic I have an older Rakudo checkout. I'll upgrade in a bit.
moritz remember to rm -rf t/spec/ to allow it to be replaced by a git checkout 18:40
chromatic That I had, fortunately. 18:41
So far so good, as the sanity tests all pass. 18:48
If I'd broken hashes, Rakudo probably wouldn't start correctly. 18:49
whiteknight I definitely think we can still squeeze more performance out of our hashes 19:00
maybe not a hell of a lot more, but there's still some gold to mine 19:01
chromatic We're getting to the type-safe hash functions on the performance list. 19:03
plobsing ~~ 19:05
chromatic moritz, dukeleto, the only failure I saw is t/spec/integration/99problems-41-to-50.rakudo 19:09
pmichaud 41-to-50 is a known failure. 19:22
We should fudge it out for now.
pmichaud does that.
chromatic Excellent. 19:23
pmichaud Done. 19:24
chromatic t/spec/S05-metasyntax/repeat.rakudo #14 was a passing TODO for me too
dalek ast: 39de0eb | pmichaud++ | integration/99problems-41-to-50.t:
Fudge out 41-to-50 gray code test until .reverse is settled.
moritz pmichaud: actually it's a regression... you should file a ticket if you just fudge the test 19:30
pmichaud moritz: I wasn't convinced the original test was correct. 19:34
moritz pmichaud: I asked TimToady
pmichaud feel free to file a ticket then :)
moritz irclog.perlgeek.de/perl6/2010-09-17#i_2837846 19:35
dalek rrot: r49089 | chromatic++ | trunk/src/hash.c:
[Hash] Extracted two funcs from parrot_hash_put.

given a STRING. parrot_hash_store_value_in_bucket() is a reusable way to store a value in a bucket after you've already fetched it. Both of these functions are static. If you ever use them outside of this file, you will incur the wrath of premature optimization.		 19:37
rrot: r49090 | chromatic++ | trunk/src/hash.c:
[hash] Optimized Parrot_hash_thaw().

what?" functions from the previous commit in this hot path improves hash thawing for the most common case dramatically and Rakudo startup by some 2.5%.
moritz is pir's inc $I0 in-place? 19:39
whiteknight chromatic: I agree with you about the "pre-alpha" nonsense. What I suppose they meant to say was something along the lines of "preview" or "pre-release" 19:42
or "first public display of code"
chromatic We already have good words for all of those things though. 19:43
"release"
purl "release" is probably pretty moot when only you and people that hack on it are using it
whiteknight chromatic: it's not really a release though
not as I understand it
they're just making their code public for the first time
chromatic Sure, but my argument is why bother? 19:44
whiteknight well, if it's an open-source project, the source needs to go "open" eventually
that doesn't make it a release
the whole statement on their part of it being a "pre-alpha" release was screwey. They used a weird buzzword when they should have said something completely different 19:45
chromatic If people can't use your code, there's no point in releasing-it-but-not-really.
That's pretty much a marketing game to prove that your code exists.
"Our code is worthless, but it does exist! Hooray! Punch line starts over there."
whiteknight they definitely should not have used the word "release". That is obviously a falsehood 19:46
but you can't blame some over-eager, under-experienced college kids to get all the PR terminology right
especially not on their first pre-alpha press release :) 19:47
chromatic No, they're just copying the prevailing... what's the opposite of wisdom?
whiteknight "Windows"
purl "Windows" is a trademark. /.*windows.*/ is not
chromatic The sound and fury of crowds.
whiteknight cacaphony?
cacophony* 19:48
chromatic I like the alliteration.
20:04 whiteknight left
dalek kudo: a90ae93 | pmichaud++ | src/Perl6/Compiler.pir:
Restore '-c' option to check a program's syntax without executing it.
 20:33
kudo: 9e46fe0 | pmichaud++ | src/core/Any-list.pm:
Set Any.reverse to flatten the invocant before reversing.
ast: 1210d8f | pmichaud++ | integration/99problems-41-to-50.t:
Unfudge gray() test in 99problems-41-to-50.t .
kudo: a204ba8 | moritz++ | src/core/Str.pm:
avoid calling .subst in str2num-rat
 20:39
ast: 5b48fa0 | moritz++ | S05-metasyntax/repeat.t:
unfudge test for catching p5 style general quantifiers in regexes
plobsing chromatic: Parrot_hash_thaw, as an optimization opportunity rakudo, is mostly mined out. Most hashes thawed have 2 or fewer entries, and therefore, the loop behaviour is not that which dominates. Only deciding what is to be done matters there. 20:43
I even cheated and cached the vtable entry to call to no avail
jnthn plobsing: ooc, what's in those thawed hashes? 20:44
plobsing String => Int for lexpads.
a non-obvious optimization PCT could do is to lift lexicals to higher scopes where legal to bunch lexicals together more 20:45
that would create fewer hashes with more content, which should be faster
chromatic plobsing, the function calling overhead is actually significant there.
plobsing chromatic: which function calls? 20:46
chromatic Parrot_ImageIO_shift_integer and Parrot_ImageIO_shift_string.
plobsing that's how I cheated 20:47
fetch_entry = info->vtable->shift_{integer,string,pmc}
jnthn plobsing: ah, OK, makes sense.
chromatic Did you mark it const? 20:48
plobsing no. I could try. but it only gets assigned at the top, and the assembly doesn't appear to be reloading it.
chromatic My thought was to put all of these functions in their own source file and let a smart compiler figure it all out. 20:49
That'd make a huge mess though.
plobsing can't mark it const. apparently assigning in different branches of a switch don't count as single assignment only 20:50
chromatic You could make a fetch_string, fetch_integer, etc. 20:52
Even only optimizing for the STRING/int case is significant.
That's ~12% of startup time for Rakudo. 20:53
10.75% now 20:54
nopaste "plobsing" at 192.168.1.3 pasted "[PATCH] vtable cheating" (79 lines) at nopaste.snit.ch/23419 20:55
chromatic To make it const, cheat: make a static function which returns the right value. 20:57
dalek rrot: r49091 | mikehh++ | trunk/ext/nqp-rx/src/stage0/HLL-s0.pir:
fix codetest failure - tidy up pod syntax (this has been pushed upstream)
 21:02
21:04 bluescreen left
jnthn chromatic: 10.75% of our startup time is on hash thawing? 21:06
21:08 whiteknight joined
plobsing chromatic: did the const thing. didn't help much. 21:16
chromatic yes. 21:20
pmichaud this morning I discovered that rakudo startup accounts for 22.6% of "make spectest" 21:26
chromatic It's less now. 21:28
pmichaud starts another set of measurement runs to find out. 21:29
21:34 davidfetter left
mikehh All tests PASS (pre/post-config, make corevm/make coretest, smoke (#85), fulltest) at r49091 - Ubuntu 10.10 beta {g++-4.5 with --optimize) 21:42
21:42 davidfetter joined 21:45 hercynium joined
luben good evening people 21:53
whiteknight hello luben
luben chromatic++ # for hash thaw optimizations 21:55
21:55 patspam left
luben I do not think that our main problem now are hashes - we could squeeze max 5-10% perfirmance improvement from optimizations in hashes 21:59
If we make a better GC we could improve our performance orders of magnitude
plobsing ImageIO sucks. it converts between pointer into buffer and index within buffer and back for every item thawed. 22:01
luben may be this is too optimistic, gut working on the GC will pay better in terms of performance
plobsing that's why it shows up so high in the callgrind costs 22:02
luben plobsing, you are right 22:03
22:04 ruoso left
plobsing it needs rewriting. either to play nice with strings and be dog slow, or to cheat more with strings and be much faster. 22:05
22:09 pjcj left, preflex left, pjcj joined 22:10 cosimo left, cosimo joined, bluescreen joined 22:12 contingencyplan left
luben plobsing, about this dilema - there are talks, that we are going to refactor STRING storage, so cheating will be very temporal solution 22:14
We will need to change that again afres STRING storage refactor
now 75% of the hash_that is in ImageIO_shift_string/integer 22:16
22:16 bluescreen left 22:17 preflex joined
chromatic If a couple of hours of cheating now gives us a couple of percent of performance, that's worth it. 22:19
plobsing luben: ultimately, I want to get thawing packfiles completely away from strings. they're a PITA.
luben plobsing, I have same idea
jnthn I did some packfile stuff int he past and never really liked the use of strings there either, fwiw.
luben couldn't we rewrite it in terms of ByteBuffer
plobsing luben: um no. maybe a custom OpcodeTBuffer 22:20
jnthn I think the important thing is that whatever it's accessed as, it should be able to be pointed at a chunk of mmaped memory.
plobsing but another layer is another layer. it will be slower.
luben jnthn, +1 about mmaping 22:21
nwellnhof yeah, we have to cut down on layers.
what parts of the string API does packfile thawing use? 22:22
plobsing it just kinda begrudgingly uses the Buffer/STRING struct
it gets no benefit from strings 22:23
nwellnhof but that shouldn't be a performance problem
plobsing it is when they can be relocated under you and you have to keep checking against that 22:24
I'm rewriting to use pinned strings
the common case is the strings being external, which never move
22:27 baest_ joined
dalek rrot: r49092 | nwellnhof++ | trunk (9 files):
[gc] Command line option for dynamic GC threshold

could be freed but are not yet collected to a percentage of total memory that is actually needed. Default is 25, maximum is 1000. Increasing the dynamic threshold results in fewer GC runs and more memory consumption. Usage: parrot --gc-threshold=50 parrot --gc-threshold=200		 22:27
22:27 bluescreen joined
whiteknight nwellnhof: There was a ticket somewhere that requested that 22:28
nwellnhof was there?
dukeleto aloha, msg chromatic the stress_strings.pir slowdown happens after 2.7.0 and before r49088 22:29
aloha dukeleto: OK. I'll deliver the message.
cotto and parseflags_minimal becomes less so
22:30 preflex left
nwellnhof cotto: yeah, this is all a bit ad-hoc 22:30
but i can fix it if someone tells me how
dukeleto aloha, msg chromatic smaller range : the stress_strings.pir slowdown happens after 2.7.0 and before r48758
22:30 cosimo left
aloha dukeleto: OK. I'll deliver the message. 22:30
22:30 baest left, cosimo joined
sorear aloha, tell p6eval this is a test of cross-chatnet messaging 22:31
aloha, msg p6eval this is a test of cross-chatnet messaging
aloha sorear: OK. I'll deliver the message.
sorear: Okay.
sorear: OK. I'll deliver the message.
sorear: Okay.
nwellnhof dukeleto: try stress_strings.pir with r49092 and the new --gc-threshold option
dukeleto nwellnhof: what is --gc-threshold ? 22:32
whiteknight nwellnhof: TT #827
I think --gc-threshold that you just implemented may satisfy that 22:33
sorear nqp: say(2+2)
p6eval nqp: OUTPUTĀ«4ā¤Ā»
whiteknight (or, come close)
nwellnhof it's a new command line options. r48585 changed that value from 50 to 25.
22:33 preflex joined
dukeleto nwellnhof: interesting 22:33
nwellnhof whiteknight: i think #827 is a bit different. it's about the total amount memory ever allocated. 22:34
gc-threshold is more like the relative amount of memory wasted
time ./parrot --gc-threshold 25 examples/benchmarks/stress_strings.pir 22:35
real 0m6.912s 22:36
time ./parrot --gc-threshold 100 examples/benchmarks/stress_strings.pir
real 0m5.352s
luben nwellnhof, ./parrot /usr/src/rakudo/perl6.pbc -e '' segfaults here after gc-treshild commit 22:38
sorear I am thinking that tell does not work at all ...
luben amd64,gcc-4.5 (now compiling with gcc-4.4) 22:39
nwellnhof luben: can you tell where it segfaults?
luben nopaste comming 22:40
pmichaud with r49091, rakudo startup takes 22.8% of total spectest time (a greater percentage, but less overall time than r49074 took)
nopaste "luben" at 192.168.1.3 pasted "segfault" (34 lines) at nopaste.snit.ch/23421
luben this is on linux amd64, gcc-4.4 22:41
pmichaud overall spectests are 1.4% faster (17 seconds) with r49091 than r49074.
(comparison between r49074 and r49091 could be different due to other factors.) 22:42
(so might not be terribly meaningful at this level)
nwellnhof luben: that's strange. did you rebuild rakudo?
luben yes
I'll try again
ops, my fault... rakudo was reading installed files 22:43
sorry for the false alarm
nwellnhof no problem
22:46 patspam joined, patspam left
luben my fast tests here showed that default gc-threshold of 25 is too conservative 23:02
here is a quick chart luben.spnet.net/gc-threshold.png 23:04
measured is rakudo startup time
nwellnhof luben++ # for his fast charting skills 23:05
a more GC-heavy benchmark would be more interesting though 23:06
luben what i could test ?
nwellnhof maybe examples/benchmarks/stress_strings.pir 23:07
luben ok. coming in a minute 23:08
plobsing luben: is that benchmark time? how does memory consumption go?
luben plobsing, yes, this is time... not measured mem consumption
nwellnhof is there an easy way to measure maximum memory consumption?
plobsing it's a memory consumption/cpu time tradeoff. knowing both curves is important 23:09
dukeleto nwellnhof: my benchmarking script doesn't allow for passing command line args to some but not others, so gc-threshold itsn't going to help me :( 23:10
plobsing nwellnhof: probably read from somewhere under /proc (linux only) 23:11
nwellnhof plobsing: yes, but ideally you would have to poll that somehow before every GC. 23:13
luben luben.spnet.net/gc-threshold-string.png
here is the new benchmark
fothe new benchmark is time for execution of examples/benchmarks/stress_strings.pir 23:14
plobsing nwellnhof: why? can't we simply check that at the end? 23:17
sure GC *can* release memory back to the system, but it rarely does. especially in a simple benchmark that just checks memory consumption of startup 23:18
nwellnhof plobsing: accurately measuring memory usage is really tricky. the GC does release string memory all the time, for example. and i'm not really interested in startup benchmarks here. 23:19
i could simply use the internal GC stats, of course. but it would be nice if i could compare them to real statistics from the kernel. 23:21
plobsing nwellnhof: write a kernel module? 23:24
luben ok, another quick and naĆÆve stats measuring speed/mem usage of simple perl6 program - "sleep 10" is comming in a minte 23:25
nwellnhof plobsing: the data in /proc/self/smaps should be enough. especially the "dirty" values. 23:26
bacek aloha, humans 23:28
23:29 kid51 joined
luben here is the new chart - the blue time is execution time in ms, the red line in resident size (RSS) MB * 6 (in order to plot them on the same meaningfull chart) 23:33
luben.spnet.net/gc-time-mem.png
whiteknight aloha, bacek 23:37
nwellnhof luben: is the last one for rakudo startup or stress_strings? 23:38
luben rakudo startup
bacek hi whiteknight 23:39
luben I could not find how to measure process max mem consumption
whiteknight bacek: hat string_gc_encapsulate branch did get merged to trunk? 23:40
bacek whiteknight, yes
23:40 Paul_the_Greek joined
whiteknight okay, it looked like there was some confusion 23:40
Paul_the_Greek Good evening, folks-for-whom-it-is-evening.
whiteknight good work on that, by the way
hello Paul_the_Greek
Paul_the_Greek Hey whiteknight.
purl whiteknight is mailto:wknight8111@gmail.com or the grand master funk or wknight8111.blogspot.com/
Paul_the_Greek whiteknight: What do you think about testing for Inf/NaN when converting floats to integers? 23:42
bacek meh... Bloody svn...
whiteknight Paul_the_Greek: I have to believe that the particular operation of converting the two happens sufficiently rarely that we can add an extra check there without kiling performance 23:43
Paul_the_Greek Probably true. I think it might be tricky to find every place we do it.
luben bacek, indeed 23:44
23:46 zostay left
Paul_the_Greek 179 occurrences of (FLOATVAL) in the system. 23:46
mikehh bacek: ...not for very much longer ... :-} 23:48
23:49 davidfetter left
bacek holy shit... git-svn committing merge as set of individual commits from trunk... 23:51
dalek rrot: r49113 | bacek++ | branches/gc_massacre/runtime/parrot/library/pcre.pir:
[pcre] Add library name genned by install of pcre 8.10 from source.
 23:52
bacek oookey. I'll fix it later. Time to go.
rrot: r49114 | bacek++ | branches/gc_massacre/DEPRECATED.pod:
Add deprecation notice for fixed_8 encoding.
rrot: r49115 | bacek++ | branches/gc_massacre/src/packfile/pf_items.c:
Fix r48994
rrot: r49116 | bacek++ | branches/gc_massacre/src/pmc (2 files):
implement Boolean init_int vtable and use it in FixedBooleanArray
rrot: r49117 | bacek++ | branches/gc_massacre/config/init/hints/mswin32.pm:
When using MSVC, always use lib.exe for static libraries
rrot: r49118 | bacek++ | branches/gc_massacre (2 files):
Use fixed size allocator for small hashes
rrot: r49119 | bacek++ | branches/gc_massacre/src/pmc/bytebuffer.pmc:
reintroduce ByteBuffer string result validation deleted in r48889, the catching tests were passing for wrong reasons
rrot: r49120 | bacek++ | branches/gc_massacre/runtime/parrot/library/URI/Escape.pir:
Use find_encoding instead of find_charset in URI::Escape
 23:53
rrot: r49121 | bacek++ | branches/gc_massacre/src (2 files):
Don't use STRING_equal yet
rrot: r49122 | bacek++ | branches/gc_massacre/docs/project/release_manager_guide.pod:
pencil in mikehh for the Feb release. He volunteered and didn't specify. 3.0 is going to be a biggy, so I'm hoping somebody specifically requests it
mikehh hey what happened to r49093 - r49112 23:56
sorear dalek didn't report them, but they exist in the repo 23:59

Powered by Cro and the RakuĀ® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!