Parrot 4.2.0 "Ornithopter" | parrot.org | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 10 April 2012.
dalek rrotStore: 84feff6 | Whiteknight++ | mysql/pmc/mysqldata (2 files):
[MySQL] Add in a few more vtables so we can iterate over a table, and loop over row elements
 00:28
rrotStore: 0605f32 | Whiteknight++ | mysql/pmc/mysqldata (2 files):
[MySQL] Add in a few interfaces for iteration. MySqlDataTable is its own iterator, so implement that interface. MySqlDataRow uses a HashIterator over the column names hash. In the future we might use something more custom, since rows should be able to be iterated by column name and by raw indexed value
rrotStore: 77caf91 | Whiteknight++ | .gitignore:
Hide the dynext directory in .gitignore
whiteknight more or less, we have MySQL bindings on Parrot now
00:37 contingencyplan joined 00:47 contingencyplan joined 00:49 jsut joined 00:54 jsut_ joined 00:59 jsut joined
dalek rrotStore: f6489a7 | Whiteknight++ | mysql/pmc/mysqld (3 files):
[MySql] Condense .query and .query_no_results into a single method that returns a table for a select, and returns an integer count for insert, update and delete for number of rows affected. Several other code cleanups.
 02:04
02:04 whiteknight left 02:51 plobsing joined 04:26 l3l1p joined 06:49 fperrad joined 07:10 fperrad joined 07:18 aloha joined 07:19 perlite joined 07:55 jevin joined 08:22 fperrad joined 08:25 dngor_ joined 08:36 _mayank joined 08:41 elmex joined 09:13 rich joined
dalek kudo/nom: 64f70e8 | jnthn++ | src/Perl6/Grammar.pm:
Fix a thinko.
 09:51
kudo/nom: 0227a99 | jnthn++ | src/Perl6/World.pm:
Remove a workaround I couldn't figure out why was needed; turns out it was just the buggy call from token subset.
 09:59
kudo/nom: 1639834 | jnthn++ | src/Perl6/Grammar.pm:
Eliminate the last remaining use of parse_name in the Grammar.
 10:14
kudo/nom: e07d6e4 | jnthn++ | src/Perl6/Actions.pm:
Toss an unused action method that mostly dates back to the pre-nom days.
kudo/nom: a803152 | jnthn++ | .gitignore:
Add .exp and .lib to .gitignore.
Heuristic branch merge: pushed 630 commits to parrot/ayardley/documentation_revisions by ayardley 10:19
kudo/nom: a648357 | jnthn++ | src/Perl6/Actions.pm:
Eliminate the majority of the remaining uses of parse_name; just one left in Actions now.
 10:30
rrot: c245d17 | alvis++ | README:
There is no 'NEWS' ([t]here's the truth of the signal, and then there's the puppet theatre .... :).
 10:35
10:38 lucian joined 10:42 whiteknight joined
whiteknight good morning, #parrot 10:42
nine Good morning, whiteknight
tadzik hello #parrot 10:51
nine Hi tadzik
10:52 JimmyZ joined
dalek kudo/nom: 8f94276 | jnthn++ | src/Perl6/ (2 files):
Eliminate last remaining parse_name call, whihc was in compilation of variables. Another tiny speedup from avoiding the reparsing.
 11:05
kudo/nom: 1421472 | jnthn++ | src/Perl6/Grammar.pm:
Toss parse_name.
rrot: 446acba | alvis++ | docs/project/release_manager_guide.pod:
Added myself as a Release Manager for the June release.
 11:07
nine whiteknight: another data point. Sharing the interp lock between all threads, thereby serializing access to all GCs does not help at all, indicating that the problem is not concurrency related. 11:10
whiteknight nine: You said you found the place where the PMC is initially allocated?
nine whiteknight: no. It's pretty much random PMCs. Minor variations in test data or parrot's code change which PMC is found. I've seen the "sayer" sub, callcontexts, an Integer and even the other thread's interp. 11:12
JimmyZ last heard 6model will be in parrot core before parrot 4.0, but .... 11:13
nine whiteknight: from all I've seen I'm pretty sure that somehow threads share some parts of the pointer arrays but all attempts to produce evidence for this failed so far. 11:14
whiteknight in src/gc/gc_gms:gc_gms_allocate_pmc_header, why do we lock after gc_gms_maybe_mark_and_sweep? 11:15
It seems like we should lock before that, right?
nine whiteknight: gc_gms_maybe_mark_and_sweep does the locking itself 11:16
whiteknight: IIRC I did it this way because mark_and_sweep can be called from outside the GC itself.
whiteknight ok 11:17
nine In the long run I want to get rid of these locks anyway... 11:19
11:31 schmooster joined
nine whiteknight: disabling the reuse of removed cells in pointer arrays makes the problem disappear! 11:40
whiteknight that....seems weird
so it's a problem with pointer arrays?
nine whiteknight: well it would fit with everything I've seen so far. Although I'd like to be cautious here. I've already wrongly suspected pointer arrays once. 11:42
whiteknight: maybe it's not the pointer arrays themselves but some call to Parrot_pa_remove which is wrong. So far we've concentrated only on Parrot_pa_insert 11:44
whiteknight ah, that's a good point
nine whiteknight: this is it! 12:08
whiteknight: write barriers!
whiteknight: doing a PARROT_GC_WRITE_BARRIER on a shared PMC (which is owned by the main thread, not the current one) causes it to be removed from another interp's objects list and put on the wrong interp's dirty list 12:11
whiteknight: and PARROT_ASSERT_INTERP didn't catch this because it explicitely ignores shared PMCs since Parrot_gc_mark_PMC_alive knows how to hande those. 12:13
whiteknight nine: I thought write barrier just set a flag 12:15
nine whiteknight: no, in addition to setting the flag it does a Parrot_pa_remove(interp, self->objects[gen], item->ptr); followed by item->ptr = Parrot_pa_insert(interp, self->dirty_list, item); 12:16
whiteknight oh man 12:17
nine whiteknight: I think the flag is just to not repeat this for already dirtied objects
whiteknight I might have never thought to look there 12:18
nine I added a PARROT_ASSERT(Parrot_gc_pool_is_owned(interp, self->pmc_allocator, item)); to every place where Parrot_pa_remove is called. The stacktrace told the rest of the story
whiteknight very clever
I bet performance tanked with that assertion active 12:19
nine Everything makes so much sense now. It takes some time for the bug to appear because it only happens with objects in older generations
oh yes it did :)
12:19 bacek joined
nine bacek: I just found the source of my GC/threading problem :) 12:20
whiteknight nine: commit a fix and I'll test it here too 12:21
nine The question remains: how can I actually fix it?
whiteknight oh yes, I guess that's the first thing to do
Parrot_pa_remove(pmc->orig_interp, ...) 12:22
nine Using orig_interp for anything but assertions can only be the very very last resort
whiteknight if (!PObj_is_shared_TEST(pmc)) { ... mark dirty ... }
nine But then the object wouldn't be marked dirty even though it should be. May even be that it fixes the test program, but I'd bet that extremely hard to debug problems would still remain... 12:24
Well, we don't really need orig_interp since shared PMCs can only belong to the main interp. Storing a pointer to the main interp in a global variable or the parent_interpreter or something like that would suffice. 12:27
whiteknight We do store an "emergency interp" somewhere for generating backtraces
If we populate that with the main interp reference, we can probably use that 12:28
src/interp/api.c static Interp* emergency_interp = NULL; 12:30
We can add an accessor for that, and make sure it gets set to the main interp only
nine I don't really like the name though. It sounds very different from what I'd be using it for. 12:31
whiteknight we can rename it
it was only put in for backtraces, in cases where things blow up and we don't have an interp reference handy
but we can rename it to "main interp", and use it for other things 12:32
or you can add your own variable too.
nine Finding a way to remove the lock from gc_gms_write_barrier will be a very interesting exercise... 12:38
12:43 PacoAir joined
whiteknight "interesting" is a fun word for it 12:43
nine well I already got an idea :) I could add a second dirty_list to the GC where access is protected by a lock same as I did for the block_gc_mark counter. All accesses by other threads have to go to the protected counter/list while the owning thread may use the lock free original counter/list. The Parrot_pa_remove(interp, self->objects[gen], item->ptr); would have to go though. Maybe I can move this to mark_and_sweep where I hold the locks 12:46
This way a single threaded program only has to care about locking during the GC run which is done sparsely anyway
alloc_test.pir is running stable for five minutes now with constant memory usage :) No sign of stopping 12:48
dalek kudo/nom: 99e7750 | moritz++ | src/ (3 files):
try to silence some compiler warnings (RT #112310, #112308)
whiteknight nine++
benabik nine++ 12:56
dalek rrot/threads: ab1d2d4 | nine++ | / (3 files):
Perform GC write barriers for shared PMCs only on the main interp

Performing write barriers on a different interp than the one owning a PMC leads to corruption of the GC's objects and dirty lists since the write barrier removes the PMC from the objects list of the generation the PMC belongs to. This causes the next_free pointer of the main thread's GC's objects list to point to a place in the child interp's objects list.
Fixed by checking the shared flag and delegating the call to the main_interp if the current interp is a child.		 13:23
13:23 kid51 joined
rrot/threads: 145ba94 | nine++ | src/thread.c:
MUTEX_INIT the interp_lock
13:26 mdupont joined, mdupont_ joined
dalek p/cstruct-work: 7ff1038 | (Arne Skjærholt)++ | src/6model/reprs/CStruct.c:
General cleanup.
 13:38
p/cstruct-work: f6a394e | (Arne Skjærholt)++ | src/6model/reprs/CStruct. (2 files):
Implement getting reference type members in CStruct.
nine threads_playground seems to be very, very stable now :) It just runs and runs... 13:57
whiteknight awesome 13:58
13:58 benabik joined
whiteknight if the single-threaded case continues to work, passes all tests and doesn't have a big performance degredation, and if we can add windows support, I think we can start talking merger 13:59
dalek kudo/nom: f524138 | jnthn++ | src/core/Str.pm:
A few little optimizations to some Str builtins. Found by looking at profile of code from masak++; seems to shave a few percent off it.
 14:00
14:07 JimmyZ_ joined
dalek kudo/nom: de1eaf5 | moritz++ | src/core/Str.pm:
remove redundant conditional in Str.match-list
 14:15
rrotStore: e0a7dc1 | Whiteknight++ | Makefile:
Add mysql to the make clean target
 14:16
rrotStore: b1d907e | Whiteknight++ | Makefile:
Re-add mongodb to the build. Fix build for mongodb and mysql so the shared libs always get built.
rrotStore: a09b843 | Whiteknight++ | / (2 files):
Make sure we include mysql in the install. Update readme with information about the three current projects, how to get prereqs, and how to build/install them
rrotStore: 32146e0 | Whiteknight++ | / (4 files):
[MongoDB] Several fixes so we can create a simple BsonDocument and save it successfully to the DB
rrotStore: 387bbb5 | Whiteknight++ | mongodb/pmc/mongodbcontext.pmc:
[MongoDB] Add a get_bool to determine if the db is connected. Add a disconnect method to disconnect
benabik What is this ParrotStore?
14:16 nbrown joined
benabik I mean, can I buy an African Blue? 14:16
Hm. Three disconnected database drivers? 14:18
JimmyZ wonders what difference is between ParrotStore and libffi which can conect mysql 14:20
benabik JimmyZ: It seems to present MySQL as actual objects instead of opaque pointers you have to pass to FFI functions. 14:21
JimmyZ benabik: as a user, I'm not aware of it, 14:22
tadzik as a user of what? 14:23
JimmyZ who connect mysql through parrot
nine Well except for some codingstd fails and expected task fails everything looks fine :) 14:24
off to a Rhapsody concert now :)
tadzik oh, concerts 14:25
benabik JimmyZ: I think ParrotStore is new. First commit appears to be Apr 12, 2012 14:26
JimmyZ benabik: yeah 14:28
dalek kudo/nom: d6cd1e2 | jnthn++ | src/core/MapIter.pm:
When I added FIRST/NEXT/LAST, it was idiomatic but not quite so fast. This makes it faster. Another little bit of masak++'s program.
 14:35
14:47 preflex_ joined 15:14 jsut_ joined 15:19 jsut joined 15:28 jsut_ joined 15:33 jsut joined 15:38 jsut_ joined
dalek p: 3a23406 | jnthn++ | src/6model/reprs/CStruct.c:
Don't try to allocate 0 bytes in CStruct REPR.
 15:43
15:43 jsut joined 15:55 jashwanth joined
whiteknight benabik: Yeah, it's just a bunch of disconnected DB bindings 16:00
I might build up from there, but I wanted to add bindings and I didn't want to create three separate repos 16:01
16:11 plobsing joined 16:29 l3l1p joined 16:32 alvis joined 16:52 jsut_ joined 17:04 l3l1p joined 17:08 jsut joined 17:35 benabik joined 17:44 dmalcolm joined
dalek kudo/nom: 9df76e9 | jnthn++ | src/ (3 files):
Implement various cases of binding into pseudo-packages.
 17:55
kudo/nom: 442db9f | jnthn++ | t/spectest.data:
Turn on S02-names/pseudo.t.
18:16 lucian joined
dalek rrotStore: 1169764 | Whiteknight++ | / (3 files):
[MySQL] Read out the rows from the result set eagerly, MySQL doesn't seem to intend for the result objects to be long-lived
 18:33
rrotStore: d3c4d95 | Whiteknight++ | mysql/pmc/mysqldbcontext.pmc:
[MySQL] Support multiple results from multi-statement queries or from stored procs. A few other small code cleanups
18:36 brrt joined
Coke ugh, I hate the mentor GSOC mailing lists. 19:43
moritz I mostly just ignore it these days 19:44
except root-level mails from Carol
benabik The student list isn't much better.
Coke IWBNI people speaking on behalf of google were marked as "moderators". I think that would help slightly. 19:46
20:04 lucian joined 21:04 mdupont joined 21:20 perlite_ joined
aloha (parrot/parrot) Issues opened : 756 (Parrot Build Fails at Build Test) by gtoast : github.com/parrot/parrot/issues/756 21:40
21:43 kid51 joined 22:06 gtoast joined
gtoast Seems like the tutorial here(www.parrotblog.org/2008/03/targetin...t-vm.html) is a bit out of date. Is there an updated version? 22:07
As is the auto-generated tutorial.
22:09 kid51 joined 22:17 nbrown joined 22:42 contingencyplan joined 23:03 kid51 joined 23:30 davidfetter joined 23:54 gtoast joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!