Parrot 4.7.0 "Hispaniolan" | parrot.org | Log: irclog.perlgeek.de/parrot | #parrotsketch meeting Tuesday 19:30 UTC
Set by moderator on 4 September 2012.
00:08 benabik joined
whiteknight rurban++ 00:13
I think threads will keep getting better and better with these kinds of updates
rurban I've also implemented now a cmdline option --numthreads and Parrot_set_num_threads(INTVAL) for run-time and rakudo. 00:15
If I only could repro the remaining failures. I gave away my mac air to my wife 00:16
00:50 benabik joined
rurban ./parrot --numthreads 4000 t/src/threads.t nive 01:09
nice 01:10
01:10 Coke joined
dalek rrot/threads: fba6c79 | rurban++ | / (9 files):
[GH #830] Implement platform Parrot_get_num_cpus

Parrot_get_num_cpus is the new default instead of MAX_THREADS. Initialized at startup		 01:15
rrot/threads: b6c1cda | rurban++ | / (7 files):
[GH #830] Add Parrot_set_num_threads and --numthreads cmdline

The new parrot cmdline option --numthreads overrides Parrot_get_num_cpus(). Parrot_set_num_threads is called by Parrot_interp_initialize_interpreter()
rurban is off now until tomorrow. 01:17
01:19 kid51 joined 01:22 MikeFair joined 05:10 rurban joined 07:00 zby_home joined 07:50 brrt joined
dalek rrot/threads: 3bc6b23 | dukeleto++ | t/run/options.t:
[t] Add some tests for the new --numthreads command-line option #830
 07:51
08:20 Psyche^ joined 08:53 alvis joined
tadzik msg whiteknight with the new patches threads.nqp run on parrot-nqp; still segfaults on nqp though 09:14
aloha OK. I'll deliver the message.
09:44 Ultali left 11:12 JimmyZ joined 11:19 rich joined 11:37 lucian joined 12:22 whiteknight joined
whiteknight good morning, #parrot 12:33
hmmm, I wonder if nqp is doing it's own context monkeying on dispatch? 12:40
dalek kudo/nom: 32309a4 | moritz++ | src/core/Mu.pm:
fix thinko noticed by cognominal++
 12:54
12:59 kid51 joined 13:24 PacoAir joined 13:45 benabik joined
dalek p: 872ecdb | jnthn++ | src/NQP/ (2 files):
Add an export trait for packages and get it to do the required installation.
 14:17
p: 3fac7df | jnthn++ | src/stage0/ (9 files):
Update bootstrap so we can use export updates in the NQP source itself.
p: 2ea2f89 | jnthn++ | src/QRegex/Cursor.nqp:
Clear up EXPORT hackery in favor of the new 'is export' trait.
14:34 rich left
dalek p: 1d3486a | jnthn++ | src/NQP/Actions.pm:
Implement is export on routines.
 14:48
p: bf091cc | jnthn++ | src/NQP/ (2 files):
Implement import of exported symbols upon use.
kudo/nom: 78efb65 | jnthn++ | / (2 files):
Get latest NQP with better export handling, enabling elimination of a hack.
 15:02
15:23 particle left 15:25 alvis_ joined 15:28 lucian joined 15:29 JimmyZ joined 15:41 pmichaud joined 15:42 not_gerd joined
benabik diakopter reported built failure on Activeperl/MSVC2010: gist.github.com/3732946 16:01
(in #perl6)
pmichaud I'm suspicious of github.com/parrot/parrot/commit/b6...335956253, line 143 of src/dynpmc/Rules.in 16:07
it's using $< in a makefile rule, which nmake doesn't seem to support.
not_gerd see msdn.microsoft.com/en-US/library/cb...80%29.aspx 16:14
$< is valid only in inference rules 16:15
pmichaud that would do it. 16:16
on a somewhat related topic, it bothers me a bit that os.pmc in src/dynpmc/os.pmc declares an 'OS' PMC... that would seem to possibly conflict with the 'OS' PMC that is now defined in core Parrot. 16:17
feels like the one in dynpmc ought to be 'OSdummy' or something like that.
16:27 particle joined
dalek kudo/nom: 5aa57b9 | jnthn++ | src/ (2 files):
Don't try to inline routines we know got wrapped before CHECK time.
 16:40
rrot: 76badef | benabik++ | src/dynpmc/Rules.in:
dynpmc/Rules: Revert OS build rules to pre 97aa16a

The rule to build os.str seems to be unnecessary and breaks nmake.		 16:54
benabik pmichaud: Sadly, pmc2c wants the PMC class name to match the filename. 16:56
ttbot Parrot 76badef9 i386-linux-thread-multi make error tt.taptinder.org/cmdinfo/96020
benabik Ooops. 16:57
ls
ww
not_gerd benabik: just replace the $< with the actual path src/dynpmc/os.c 16:59
benabik Bugs me that os.pmc didn't need a .str before.
17:01 travis-ci joined
travis-ci [travis-ci] parrot/parrot#553 (master - 76badef : Brian Gernhardt): The build was broken. 17:01
[travis-ci] Change view : github.com/parrot/parrot/compare/1...badef98ff7
[travis-ci] Build details : travis-ci.org/parrot/parrot/builds/2469467
17:01 travis-ci left, zby_home joined
benabik Huh. There's already a prefix rule for .c.str 17:03
Still bugs me that the empty dynpmc needs it and apparently none of the useful ones do. 17:04
17:20 whiteknight joined
dalek rrot: 32da9dd | benabik++ | src/dynpmc/Rules.in:
os.pmc actually needs os.str

I find this somewhat bizzare, given that it's basically empty, but don't know how to fix that. Instead of using the old nmake-breaking rule, let the suffix rule in the main Makefile handle it.		 17:21
benabik benabik-- # remember make clean and reconf when testing Makefile changes
msg whiteknight You're better at the object system than I am. Is the core OS PMC and the OS dynpmc going to conflict? 17:23
aloha OK. I'll deliver the message.
whiteknight hello benabik
benabik Ohai 17:24
whiteknight ...maybe?
I think they will, yes
17:24 kid51 joined
whiteknight or maybe they won't 17:24
do a test
I *suspect* they may merge in some odd way
or the dynpmc will overwrite the built-in
benabik Hm. 17:25
benabik ponders how to test.
Hm. After using `.loadlib 'os'`, `OS.cwd()` still seems to work. 17:29
jnthn++ tested my Rule changes, but got compiler errors instead: gist.github.com/3733333 17:30
rurban I found no way that the empty dympmc overwrites the core pmc. They share the same name, so no conflict. 17:36
But thanks for the $< catch
17:36 travis-ci joined
travis-ci [travis-ci] parrot/parrot#554 (master - 32da9dd : Brian Gernhardt): The build was fixed. 17:36
[travis-ci] Change view : github.com/parrot/parrot/compare/7...da9dd26cc7
[travis-ci] Build details : travis-ci.org/parrot/parrot/builds/2469742
17:36 travis-ci left
benabik The naive expectation is that things with the same name would conflict. 17:36
rurban The .str dep was added on purpose.
Monkey patching, as in perl5
benabik rurban: Any idea what the errors jnthn got are? gist.github.com/3733333 17:37
whiteknight rurban: Can you still create a built-in OS and a dynpmc OS without conflicts? 17:38
rurban Just woke up, need my coffee first, then I'll look
whiteknight: I tested it okay. But I have to retest now.
whiteknight ok
how do you distinguish between the two types? namespacing?
rurban they share the same namespace, are both in memory. but since the dynpmc has no method the core pmc methods are used. 17:39
whiteknight okay, so they merge
rurban dynpmc's can dynamically override behaviour. later. Our os dynpmc does so, but defines no methods. Since the class name as string is used and no pointer, we are safe. At least I think so. 17:43
I have to reboot to init my vm's.
whiteknight okay, if things work reliably like that, it's a good solution to our problem 17:46
rurban
rurban++
rurban But I'll debug into it. And test windows (jontahans problem)
whiteknight jonathan has a problem? 17:55
benabik os dynpmc wont' build with activestate/msvc2010
17:55 rurban joined
rurban I also slept over numthreads. We need to move the threads from static into the heap. MAX_THREADS is a problem 17:57
whiteknight ok 18:03
18:03 contingencyplan joined
whiteknight can we make a new branch for that change? 18:03
I want to keep threads as stable as possible until a merge
rurban For the heap, yes. I think A got a thinko also, which I might have to fix. And I want nine's opinion for the heap change. 18:06
whiteknight The heap change shouldn't be too bad 18:07
Tadzik is reporting one more segfault in nqp only. I suspect one of their core types is going to need to be updated for threads 18:08
tadzik I'm not sure, but I think it's about nqp having its own LexPads
rurban I need more reboots. I renamed my logical lvm's and my qemu's. Got a bigger harddisc. 18:09
18:11 rurban joined
whiteknight tadzik: that may be. I was suspecting dispatchersub, but I remembered nqp got rid of it 18:12
I need to dig in and debug
rurban I had to add Parrot_get_num_threads() function for the scheduler 18:33
I think a make test_nqp and test_rakudo target would make sense for us. 18:35
Similar for other languages (downloading via git) 18:36
dalek rrot/threads: 0670c46 | rurban++ | / (4 files):
Protect threads_array from overflow, add Parrot_get_num_threads()

Ensure that a manual numthreads argument does not exceed the allocated MAX_THREADS. The scheduler needs to know the number of threads. Let Parrot_set_num_threads return the number of set num_threads, which might be -1 in case of errors.
Enhance the documentation.		 18:40
18:48 not_gerd left
whiteknight we have a test_all_hll target or something 19:20
rurban Coudl repro some msvc problems with dynpmc/os 19:23
MikeFair howdy all :)
dalek rrot: e482edc | rurban++ | src/dynpmc/Rules.in:
dynpmc/os: nmake fix, $< invalid for os.str

  $< is only valid for suffix rules, and also then not for nmake		 19:25
whiteknight hello MikeFair 19:33
MikeFair whiteknight: What magical beasts are you saving us from today whiteknight? :) 19:34
whiteknight looking at a few remaining issues for the threads branch merge 19:35
19:37 lucian joined
rurban I named my branch rurban/threads_array_heap 19:39
whiteknight great 19:40
19:42 TonyC joined
benabik rurban: Was that rule required? Why doesn't the generic .c.str rule work? 19:44
rurban nmake does not grok implicit suffix rules 19:45
or it does, but $< in the explicit rule is certainly wrong 19:46
benabik There are no explicit rules for any other .str files.
rurban yes. it's strange
I have even stranger linking problems with dynpmc/os. STRINGNULL and PMCNULL does not seem to be exported. 19:51
Oh, I see. cl does not allow duplicate symbols, same as HP/UX 19:54
So I need some patching for the osdummy
Eitjher patch loadlib to ignore os, or patch the resulting dynpmc/os.c 19:55
But msvc link is just too stupid. It cannot tell that there are duplicate symbols to be linked. 20:01
20:11 travis-ci joined
travis-ci [travis-ci] parrot/parrot#555 (master - e482edc : Reini Urban): The build is pending. 20:11
[travis-ci] Change view : github.com/parrot/parrot/compare/3...82edc3363d
[travis-ci] Build details : travis-ci.org/parrot/parrot/builds/2470752
20:11 travis-ci left 20:18 travis-ci joined
travis-ci [travis-ci] parrot/parrot#555 (master - e482edc : Reini Urban): The build is pending. 20:18
[travis-ci] Change view : github.com/parrot/parrot/compare/3...82edc3363d
[travis-ci] Build details : travis-ci.org/parrot/parrot/builds/2470752
20:18 travis-ci left 20:32 perlite joined
rurban I have to rewrite our dynpmc/os to dynpmc/osdummy 21:02
name clashes on windows and hpux 21:03
oops, I'm too stupid. it was a non-dynamic pmc. That's why the names clashed. With the change to pmclass OS dynpmc it works fine. Smoking msvc parrot now 21:20
smolder.parrot.org/app/projects/rep...ails/31409 with the wrong os.pmc 21:37
I also know why os.str was created. because it was a pmc, and pmc have .str files, dynpmc's not 21:42
dalek rrot: 8874c43 | rurban++ | / (4 files):
Fixed dynpmc/os windows reveiled duplicate names

Fixed several bugs in the dummy dynpmc os: The dynpmc/os was a static pmc. msvc could not link duplicate names. I had to change the pmc to dynpmc, and had to rename the os to osdummy, which is installed as os.so/bundle/dll. Tested ok on msvc.		 21:49
rurban And this is the good smoke smolder.parrot.org/app/projects/rep...ails/31410 after the fix 21:51
dalek rrot/threads: 76badef | benabik++ | src/dynpmc/Rules.in:
dynpmc/Rules: Revert OS build rules to pre 97aa16a

The rule to build os.str seems to be unnecessary and breaks nmake.		 21:52
rrot/threads: 32da9dd | benabik++ | src/dynpmc/Rules.in:
os.pmc actually needs os.str

I find this somewhat bizzare, given that it's basically empty, but don't know how to fix that. Instead of using the old nmake-breaking rule, let the suffix rule in the main Makefile handle it.
rrot/threads: e482edc | rurban++ | src/dynpmc/Rules.in:
dynpmc/os: nmake fix, $< invalid for os.str

  $< is only valid for suffix rules, and also then not for nmake
rrot/threads: 8874c43 | rurban++ | / (4 files):
Fixed dynpmc/os windows reveiled duplicate names

Fixed several bugs in the dummy dynpmc os: The dynpmc/os was a static pmc. msvc could not link duplicate names. I had to change the pmc to dynpmc, and had to rename the os to osdummy, which is installed as os.so/bundle/dll. Tested ok on msvc.
rrot/threads: ac5be43 | rurban++ | / (4 files):
Merge remote-tracking branch 'origin/master' into threads
rurban I really need to develop more on windows. It reveils stupid bugs. jnthn is right. 21:54
MikeFair rurban: FWIW I like/respect Windows for many things I think it does well :) 21:58
rurban: Or at least functionally
rurban I think there is a small security issue with loading wrong loadlibs. The name if the loaded class is not checked. I think we shoudl not fix it. But I'll create a ticket. 22:02
MikeFair rurban: You mean somebody replacing the file with something else? 22:03
rurban: Or the class within the pbc?
rurban Somebody copying a bad dynpmc/foo.so 22:04
Or from a different load-path
MikeFair rurban: yeah that's a security issue with windows regardless, I'm not totally sure how you protect against that without doing some kind of "publicKey" type thing like what the CLR does 22:05
rurban On every system. But I believe it's not worth fixing, because you cannot override a already loaded class. 22:06
You can override if you provide all methods of the old class.
MikeFair rurban: Not for the instances that already exist, but for new instances perhaps
rurban This is good for patching.
MikeFair rurban: The runtime patching of things is a seriously interesting space to me 22:07
rurban: I'm really curious to understand how erlang does it
rurban: I mean I get conceptually that they have a "current version" and new instances get that and old ones continue until they die off 22:08
rurban: I think the key is to focus on replacing/loading something with code that presents something binary compatible with a that class' "interface" hiding the guts of implementing the class itself 22:11
rurban: it's a working theory that I can replace the destination of the vtable entries with a new class implementation and the code 'should be' none the wiser (assuming children don't do bad things with implementation specific details of its parent classes) 22:13
(though I consider children making those kinds of assumptions are 'asking for it') :) 22:14
rurban github.com/parrot/parrot/issues/831 22:19
It must not be binary compatible. just respect the vtable api. provide the same methods. 22:21
found 3 threads errors on windows. smolder.parrot.org/app/projects/rep...ails/31420
windows just stopped after new P1, "Task"; schedule P1 22:23
Now I really have to load the MSVC IDE...
Uuh. I have to register my MSVC 9 Express Edition. Free but I need an Microsoft account. 22:26
The MSVC IDE is seriously superior to anything on Linux. Found a nice threading bug. 22:45
num_cpu = 1 will not work :) need at least two threads. 22:48
dalek rrot/threads: afffe06 | rurban++ | src/thread.c:
Verify that at least 2 threads are created

For a single CPU default to 4 created threads. For Parrot_set_num_threads verify at least 2, one is needed for sleep		 22:54
MikeFair rurban: You want use mine? 23:01
rurban: nm looks like you handled it
rurban What yours?
MikeFair MS account 23:02
rurban: I'm glad I'm not the only one who thinks MSVisual Studio is a nice IDE
rurban: Have you seen Lightable? 23:03
www.kickstarter.com/projects/ibdknox/light-table 23:04
he was Project Manager for Visual Studio at the time and got a vision for 'something better' 23:05
rurban I created an account, not prob.
MikeFair Just over on #Perl6 the idea came up to use Blender as an IDE/Debugger
I like the idea myself a lot 23:06
rurban Lightable looks cool, will try it out. Eclipse is on my list also.
MikeFair rurban: I'veused eclipse often enough.. I find it "confusing"
rurban: and its such a fiefdom oriented community that it's difficult to know what works with what 23:07
rurban: While I don't use emacs, I always feel like 'this must be what Emacs feels like' when I'm using it
rurban: And eclipse has many 'wrong ways' to try and do something that seem like they should work but aren't -- for instance "build" doesn't necessarily mean the same thing in all places. It could mean something different depending on the menu you selected it from -- and usually the one you want is buried a couple layers in a submenu 23:10
dalek rrot/threads: e488202 | rurban++ | src/scheduler.c:
More threads sanity checks

On single-cpu windows I had the case of NULL candidate, when looking for the thread with the fewest tasks. Bail out with "Could not find a free thread" when no free thread is found, even when Parrot_thread_get_free_threads_array_index() returned -1 (all threads allocated).		 23:11
MikeFair I appreciate and respect the complexity and flexibility they've built, but I have not been able to really completely grok it yet... though I get better everytime I use it
rurban threads are getting better and better... 23:13
wonder if darwin/x86 passes now.
MikeFair that's exciting to hear 23:14
rurban lightable uses lisp. no wonder that's it is superior. looks like an old lisp machine 23:15
MikeFair good working threading means you're a grown up VM to ppl :)
rurban: hehe - I didn't know that yet
sorear rurban: I have a darwin/amd64 machine 23:17
MikeFair rurban: Question for you about threading, aside from ease of coding in separate logic trains - is there any advantage to having 'execution threads' > 'num cpus'
sorear MikeFair: blocking i/o operations
MikeFair sorear: Well that could also be viewed a separate logic thread -- eventually that blocked thread need back on the CPU 23:18
rurban The closest visual debugger I know is adg/affinic debugger 23:19
sorear: do you want to smoke the latest parrot threads branch?
sorear rurban: sure. how?
rurban hwo many cpu's do you have?
MikeFair I'm thinking of a VM model where a thread is a virtual CPU object in the VM and everything else is a "task"
rurban git checkout threads; perl Configure; make smoke
sorear rurban: think 2, not sure 23:20
rurban I just fixed an issue with single-cpu
sorear yeah 2
rurban two might also be a corner case.
sorear I guess I need to check out parrot first 23:21
rurban with 2 cpu's we run only two threads. might be a bit too less.
sorear I have 1 package though, might confuse your code
github.com/parrot/parrot?
rurban yes
git clone github.com/parrot/parrot.git 23:22
and maybe cpan TAP::Harness::Archive
hmm, one more error in t/src/threads_io.t on windows 23:23
cleanup is also pretty slow... 23:26
sorear started make smoke 23:27
rurban thanks. I suspect one fail in t/src/threads_io.t
sorear test phase begun 23:29
rurban good. it found your number of cpu's (new code written from scratch)
sorear how do you know? would it have aborted? 23:30
or is it streaming to smolder?
rurban It's an Configure probe.
no streaming yet :)
It would have failed in auto::cpu
I haven't tested on darwin yet.
sorear t/src/embed and t/src/extend failed, t/src/threads_io passed 23:31
rurban but darwin has /proc/cpuinfo so it's easy.
interesting...
sorear cat: /proc/cpuinfo: No such file or directory
rurban also interesting. when we find no num_cpus we just use a default. 23:32
sorear Test report successfully sent to Smolder at 23:33
smolder.parrot.org/app/projects/rep...ails/31422
also I have 2>&1 | tee output for both configure and make
rurban We need someone on darwin to fix the simple embed problems. my wife arrives next week with my air.
_Parrot_new_string flat namespace. interesting 23:34
sorear how can I ask parrot to test num_cpus probing?
rurban there's no test for it yet. I jump into the debugger. gdb --args ./parrot t/pmc.task.t 23:35
b Parrot_thread_init_threads_array
sorear gist.github.com/123d73d33c08adbb5081 23:36
running just "./parrot" without gdb also dies horribly 23:37
rurban on darwin there's no rpath, yes. you'd need to set the DYLD path somehow. 23:39
libparrot.dylib not found. 23:40
sorear how?
rurban I forgot...
sorear or rather, to what?
DYLD_LIBRARY_PATH=Something ./parrot
rurban It's in blib/lib I guess 23:41
yes
dalek rrot/threads: 790f343 | rurban++ | src/thread.c:
Tune num_threads

Via API the minimum num_threads is 2. One for sleep.
Via default Parrot_get_num_cpus() use 4 as default, if
  <= 2 cpu's are found.
This leaves dual-cpu's with a bit more threads.
rurban I just gave you two more threads for dual-cpu's.
sorear Parrot_get_num_cpus returned 4 23:42
!?
rurban But your problem seems to be unrelated to threads.
hyperthreads maybe 23:43
4 is a good number
So the scheduler has something to do.
I guess your t/src/embed.t and extend problems are also related to a missing rpath resp. DYLD_LIBRARY_PATH 23:47
We seriously need to check our test libs
Interesting. On Windows t/src/threads_io.t works sometimes, sometimes not. 23:53
Looks racy.

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!