dalek arVM/eieio: 0066643 | jnthn++ | src/io/syncfile.c:
Correct comments.
 00:04
arVM/eieio: 3a0c828 | jnthn++ | src/io/syncfile.c:
Missing 'static'.
jnthn Got a bunch of code here working towards moving TTYs and pipes over to the new IO model. 00:10
But tired, so will continue with it tomorrow.
timotimo good night, jnthn! 00:25
00:34 tgt joined 00:44 benabik joined 01:35 tgt joined 02:22 tgt joined 02:25 benabik joined 02:47 ilbot3 joined 03:05 FROGGS joined 03:23 tgt joined 04:22 dalek joined 04:23 tgt joined 05:24 tgt joined 05:28 tgt joined 06:29 tgt joined 07:30 tgt joined
timotimo lowering the lexicals to locals works again 07:39
it gives up as soon as it finds a QRegex, though, because we don't scan through there yet (as it transcends Optimizer classes)
and i can see no startup memory usage in perl6-m
improvement*
i don't have to manually remove entries from lexpads in the optimizer, right? replacing all the :decl('lexical') with 'local' should be enough, eh? 07:45
maybe it's just that the overwhelming majority of lexicals that *could* be lowered to locals are actually in perl6 code rather than in nqp code 07:46
where i'm pretty sure, analysis is going to prove very tricky indeed
i should look up how exactly locallifetime is supposed to work. 07:47
yeah, i apparently have no idea how the free list works. 08:13
oh 08:15
could it be that whenever there's a free block, you just write the address of the next free block whereever whatever object used to be?
that seems to make sense
now i've got much prettier graphs 08:20
so annoying that gdb's pager pushes its "press enter to continue" right in the middle, though
dalek arVM/gdb-support: b96b7eb | (Timo Paulssen)++ | moar-gdb.py:
fix handling of free list.
 08:22
arVM/gdb-support: 61cdfce | (Timo Paulssen)++ | moar-gdb.py:
remove double histogram for now
arVM/gdb-support: 9f7631e | (Timo Paulssen)++ | moar-gdb.py:
allow more samples
arVM/gdb-support: cb98712 | (Timo Paulssen)++ | moar-gdb.py:
don't try to sample free'd objects, mark remaining null objects
timotimo gist.github.com/timo/4416af400d9c86fd0f3c 08:24
dalek arVM/gdb-support: 908cd2c | (Timo Paulssen)++ | moar-gdb.py:
don't display useless size histograms
 08:26
08:30 tgt joined 09:31 tgt joined 10:00 tgt joined
FROGGS o/ 10:21
nwc10 \o 10:22
11:18 tgt joined
jnthn timotimo++ # awesome wok on visualizing the gen2 pages!! 11:22
uh, work :)
mmm...wok...
11:58 tgt joined
dalek arVM: 068ce44 | (Tobias Leich)++ | src/6model/reprs/SCRef.c:
do nothing if sc->body already is NULL
 12:49
FROGGS that's about this btw: rt.perl.org/Ticket/Display.html?id=121253 12:51
hoelzro hi Moar hackers 13:01
I had an idea while in the shower
you know how a native application will share code/shared object pages with other applications using them? 13:02
I wonder if it would be a good idea to emulate that in MoarV
maybe it's overkill
but I just thought I'd bring it up
dalek arVM/eieio: 5c1d25c | jnthn++ | / (10 files):
Add syncstream, and move TTY over to it.

A syncstream provides synchronous access to a libuv stream, using a MoarVM DecodeStream (like syncfile does) to handle bytes -> chars. At least on Windows, things like the REPL and piping files into Rakudo work out. Note that this breaks pipes; they still need updating to the new scheme.		 13:19
timotimo hoelzro: well, at least the .moarvm files are mmapped, so they are shared between instances 13:20
FROGGS ohh cool, need to try eieio soon 13:26
timotimo FROGGS: would you like help in figuring out how to use my gdb things?
or maybe you have a specific wish?
also, have you considered activating reverse debugging? 13:27
FROGGS umm... I can test you gdb plugins, but I am not sure I can make any sense out of comparing nursery snapshots 13:28
at least that is what I think it does
timotimo hehe.
you can't even compare nursery snapshots right now
FROGGS and what do you mean aber reverse debugging?
ahh
timotimo but the plugin also pretty-prints MVMString, so that's SUPER useful! :)
FROGGS hehe, yeah
:o)
timotimo you can turn on reverse debugging and "cont", then it'll be 1000000x times slower, but afterwards you can single-step forwards *and* backwards 13:29
FROGGS ohh
that sounds interesting
and indeed very helpful
timotimo it's new in gdb7
what plugin functionality would make life easier for you? 13:30
FROGGS hmmm, let me think
I don't know yet... but since I am using gdb right now I might feel the need for something :o) 13:32
timotimo fair enough :)
FROGGS can I run shell commands within gdb? 13:35
timotimo ctrl-z, use shell, fg, use gdb :P
FROGGS because that I could just do a "make install" in moarvm and rerun my script
what? 13:36
timotimo not sure if gdb will re-load the stuff from disk when it changes, though
there is a way to give gdb a startup script that will run commands for you 13:37
like setting breakpoints and running and stuff
that could be helpful to you
FROGGS yeah, true 13:38
damn, I'm not able to create a simple case that causes the STable bug 13:40
timotimo d'oh :(
jnthn peers curiously at all the ref/unref going on in openpipe... 13:45
FROGGS ref all the things! /o/ 13:47
timotimo i don't even know what that means/does 13:48
14:02 lizmat joined
FROGGS okay, it seems like I can reduce panda to something that complains about STable 14:07
14:07 woolfy joined, woolfy left
timotimo jnthn: how about bumping MOAR_REVISION in nqp to get the intcache, then bumping NQP_REVISION + fixing the renaming fallout in rakudo? 14:07
jnthn Well, we should fix the fallout along with the bump... 14:08
timotimo yes
in the same commit, then?
jnthn Please.
timotimo will do
dalek arVM/gdb-support: 91f1240 | (Timo Paulssen)++ | moar-gdb.py:
more robust stuff, bunch up output and analysis
 14:10
jnthn FROGGS: I'm a little dubious about the process close stuff.
FROGGS jnthn: you mean that we close one end of the pipe? 14:11
s/that/where/ 14:12
jnthn When you nqp::close a handle 14:15
That you got from openpipe
FROGGS yeah
when we close our side of the pipe the child process should terminate, so we wait for that 14:16
dalek arVM/eieio: 0f70c62 | jnthn++ | / (9 files):
Move openpipe over to the new IO scheme.

With this, we're back (at least on Win32) to no spectest regressions and passing NQP tests.		 14:25
arVM/eieio: 5aa7bba | jnthn++ | src/6model/reprs/MVMOSHandle.h:
Toss two now-unused members.
 14:30
jnthn Testing on other platforms welcome
err&
timotimo will do 14:31
FROGGS jnthn: I know now how to trigger the STable thingy 14:33
jnthn: see github.com/FROGGS/STable
14:39 woolfy joined, woolfy left
hoelzro timotimo: ah, I didn't know that. cool! 14:40
FROGGS src/io/syncfile.c:39:13: error: conflicting types for ‘close’ 14:44
static void close(MVMThreadContext *tc, MVMOSHandle *h) {
/usr/include/unistd.h:353:12: note: previous declaration of ‘close’ was here
extern int close (int __fd);
same about truncate 14:45
timotimo hoelzro: ww?
hoelzro timotimo: wrt mmaping of .moarvm files
timotimo oh, yes
FROGGS so, shall we rename it to closefh and truncatefh?
timotimo hoelzro: i'm still considering unmapping the serialized portions, as they are only ever read once and then we can forget about them 14:46
hoelzro seems reasonable
timotimo jnthn: gist.github.com/timo/aad915a9cff0f0cd928d
FROGGS timotimo: I pasted that already :o) 14:51
timotimo oh, ok 14:52
thank you :)
FROGGS jnthn: this makes it compile on linux: gist.github.com/FROGGS/b320bfe9140ddf193ac9 15:09
jnthn: I did not push because you might prefer other names
jnthn Oh, how silly... :) 15:13
But yeah, those names will doo.
*do
FROGGS k 15:15
will push
dalek arVM/eieio: 15944b1 | (Tobias Leich)++ | src/io/ (5 files):
use names that work out on unix
 15:17
FROGGS [...]src/gen/m-CORE.setting 15:20
Stage start : 0.000
Unexpected named parameter 'deserialize_past' passed
at gen/moar/stage2/NQPHLL.nqp:1938 (/home/froggs/dev/nqp/install/languages/nqp/lib/NQPHLL.moarvm:add_load_dependency_task:12)
but that might even be due to bumping nqp to HEAD 15:21
ohh, yeah 15:22
jnthn Yeah, don't do that...
FROGGS that is due to a past/ast change 15:23
np, I just switch back :o)
timotimo FROGGS: aye, that's what i'm going to fix today-ish 15:30
until then, just leave the newest 2 commits of nqp out
FROGGS jnthn: btw, I've tested this on windows and it shows the STable problem: github.com/FROGGS/STable 15:31
jnthn: and it is very minimalistic too :o)
timotimo++ # :o)
timotimo oh yeah, there isn't much left :D 15:32
jnthn FROGGS++ 15:38
FROGGS jnthn: you'd just need to run the doit.bat then 15:39
so it really is about using more than one precompiled module that has a class with an attribute in it 15:45
timotimo so we accidentally say "Attribute now belongs TO MEEEEEHHHh!!!kkk"? 15:46
FROGGS I guess both attributes share the same STable, so that triggers the reposession? in the using module? 15:47
I just can guess wildly
jnthn FROGGS: argh, I thought you'd just renamed the static functions, not the vtable names... :/ 16:22
FROGGS umm 16:23
fix it?
jnthn Yeah, but it's made a huge merge conflict for the refactor I just did...
FROGGS ohh 16:24
I'm sorry
I don't even know how let the vtable names stay the same 16:25
ahh, not touching src/io/io.h should have been the right thing I guess 16:26
(besides src/io/fileops.c)
jnthn Right. 16:27
Should have only had to touch sync[file|stream|pipe].c
FROGGS yeah
jnthn Re-done it here, hopefully.
dalek arVM/eieio: 71c1650 | jnthn++ | src/io/ (5 files):
Revert "use names that work out on unix"

Renamed too much.		 16:28
arVM/eieio: 38fbf36 | jnthn++ | / (6 files):
Move generic IO-related things into io.c.

This leaves fileops.c (mostly) now just having things that only work on files.
arVM/eieio: 3962173 | jnthn++ | src/io/sync (3 files):
474fb8c | jnthn++ | src/io/syncfile.c:
 16:29
jnthn That last one that didn't get reported was putting the needed renames back
FROGGS jnthn: can you put that include back in? github.com/MoarVM/MoarVM/commit/71...7a3e1de5L6 16:30
jnthn bah, the commit message only mentioned renames... 16:31
FROGGS I'll be more careful in future 16:33
jnthn np 16:34
FROGGS it seems to build nqp right now
jnthn crosses fingers for passing "make test" in NQP 16:35
FROGGS All tests successful. 16:37
Files=89, Tests=3511, 24 wallclock secs ( 0.47 usr 0.07 sys + 21.39 cusr 1.34 csys = 23.27 CPU)
Result: PASS
jnthn \o/
FROGGS jnthn++
cool! I think I should get comfortable with the new api soon 16:38
jnthn I'm gradually clearing up 16:39
Currently looking at sockets.
Gonna make them work as far as they work on the JVM, but I can see that the socket class in Rakudo, and the JVM impl, are going to need some attention.
FROGGS .oO( jnthn will be a Socket Scientist? ) 16:40
jnthn :P 16:41
FROGGS had this in mind for two days now :o)
in the m-spectest is nothing that catches the eye either... 16:57
jnthn Nice :)
timotimo Sock Surgery! 16:59
and yes, the socket classes are kind of bad 17:00
moritz the Hackiest Thing That Could Possibly Work
timotimo :D 17:01
dalek arVM/eieio: d450827 | jnthn++ | / (16 files):
Very basic client socket support.

With some NQP and Rakudo additions also, this is enough that we can connect to some IP address and port, and get stuff from it. Missing DNS resolution so far.		 18:04
timotimo ooooh that sounds great! :D 18:06
jnthn My test was: 18:09
my $s = IO::Socket::INET.new(:host<193.200.132.135>, :port(3000));
$s.send("GET /projects.json HTTP/1.0\n\n");
.say for $s.lines;
$s.close;
That is, the one Panda uses :)
Now I "just" need to make it do host resolution... 18:10
dalek arVM/eieio: 4805a51 | jnthn++ | src/6model/reprs/MVMOSHandle.h:
Clear up some leftovers of old I/O implementation.
 18:11
FROGGS oh oh oh 18:47
jnthn++
somebody is working hard when I feed the pups :o)
timotimo pups refering to little dogs? 18:48
FROGGS in this case to my kids, yes
timotimo ah, ok
FROGGS I am musing how to implement that in v5 btw: 18:49
perl -E 'use strict; our $foo = 42; my $bar = *foo; say $$bar'
42
timotimo *foo? o_O
FROGGS that is like a reference to a slot of name foo that holds $foo, @foo, %foo and &foo in the package.WHO 18:50
so you can do that to alias subs
timotimo o_O 18:52
dalek arVM/eieio: 2ed615e | jnthn++ | src/io/syncsocket.c:
Resolve hostnames in connect.
 19:11
jnthn That means we can now run exactly the code Panda uses sockets for. 19:12
timotimo \o/
that'll definitely go in my weekly changes report :) 19:13
japhb jnthn++ # W00t! Socket code on MoarVM! 19:52
Are you going to merge eieio soon, or is there something else big that you want to land?
FROGGS japhb: there are no regression afaik, which (I hope) is half the answer :o) 19:56
nwc10 I am seeing a SEGV from t/moar/02-pipes.t on Linux and on OS X. 20:01
I'm having trouble making sense of it, because it fails differently under valgrind and just under gdb
but I end up with a NULL tc in some cases, which makes tc->loop a NULL pointer deference
this is on eieio 20:02
everything else is fine. Didn't try a rakudo spectest
FROGGS hmmm, gcc on linux does not show anything 20:10
nwc10 oh well, some progress 20:13
*(MVMint64 *)req->data = (exit_status << 8) | term_signal;
that's trashing tc
FROGGS that is my invention :/
timotimo oh whoops :D
nwc10 implying that req->data is bogus 20:14
FROGGS ahh, maybe that has changed :o)
jnthn That's the bit of code I was looking confusedly at earlier :) 20:17
nwc10 OK, so I'm stuck *how* it's happening, but req->data seems to be &tc 20:18
jnthn japhb: The sockets stuff is the last big piece, really
nwc10 so on "process exit" the callback is writing exit status information, and inadvertently wiping out the thread context
jnthn nwc10: Is this on the eieio branch?
nwc10 er, child process exit
yes, eieio
jnthn oh, you said and I missed it :) 20:19
nwc10 oh, the SPAWN macro sucks. 20:21
process->data = &result; \
it's taking the address of a variable on the C stack
FROGGS yes, result is a local variable there
nwc10 and somehow expecting it to be useful
timotimo hah, ouch 20:22
nwc10 and that address on the stack is getting written to much later
jnthn Uh...
nwc10 when it happens to be the tc parameter of a function
jnthn Oh, you just found it to...
nwc10 whoopsy
jnthn fail
nwc10 yes, fail :-)
FROGGS I thought this was already fixed in master... appearently not
nwc10 am I execused? Can I go to bed now?
FROGGS *g*
timotimo nwc10++
jnthn FROGGS: Does the error code get used anywhere when this is done with openpipe? 20:23
nwc10 hardware watchpoints for the win.
FROGGS jnthn: no, not for pipes
only for spawn and shell, which are supposed to return within the function
so, we should just set it to NULL and check for non NULL in the cb 20:24
jnthn yeah, doing that.
dalek arVM/eieio: 146ba4a | jnthn++ | src/io/procops.c:
Avoid trashing the stack with process exit value.

  nwc10++ for report/hunting it down.		 20:27
jnthn OK, I can probably merge the branch at this point... 20:49
FROGGS, timotimo: Either of you up for doing a NQP/spectest run for me?
timotimo sure
jnthn I've more to do but I don't need to do the rest in a branch, since it's gentle refactors or new additions.
I'm done ripping stuff up :)
FROGGS sure
the nqp test passes 20:50
jnthn: no new fails in m-spectest 21:00
jnthn OK, let's merge this stuff. 21:01
dalek Heuristic branch merge: pushed 31 commits to MoarVM by jnthn 21:12
FROGGS switches branches
timotimo branches switches 21:16
jnthn swinches bratches 21:17
masak stitches brambles 21:20
timotimo ooooh, tasty brambles! 21:21
FROGGS jnthn: fwiw, fetching /projects.json does work here too 21:36
timotimo YAY \o/ 21:37
moar panda for everyone!
FROGGS btw, fetching it takes 0.9s using perl6-p, which seems to split lines weirdly, and it only takes 0.58s on moar without weird splits 21:39
timotimo \o/ 21:40
jnthn just wrote the world's crappiest ever HTTP server so he can try and get server side sockets working 21:46
FROGGS hehe 21:47
timotimo IO-Socket-INET.t isn't going to work yet? 21:50
because of missing server sockets? 21:51
FROGGS correct
timotimo slowest test case ever :)
FROGGS perhaps you can spawn the server using perl6-p :o)
timotimo jnthn: do you have a good next step for me to do to get the lexical-to-local optimization bettered?
obviously i should descend into QRegex and mark any lexicals used in there as DO NOT TOUCH, instead of giving up on optimizing lexicals completely at that point 21:52
but even so, I saw lots of "yippie! i changed $foo to $bar" messages and no change in memory usage
jnthn Yeah. I guess descending onto closures and seeing what's captured is also a part of it.
I mean, a correct analysis has to do that. 21:53
Or are you doing that much already?
timotimo descending into closures?
i'm looking into any QAST::Block for usages of the lexicals
jnthn OK, and then...?
timotimo that's what makes the optimization consider a lexical "poisoned" for this specific optimization
jnthn Ah
timotimo if the lexical hasn't been used anywhere besides the very same QAST::Block they were defined in, i turn them into lexicals 21:54
jnthn Yeah, that's the incredibly conservative approach.
timotimo i turn them into locals*
jnthn *nod*
timotimo what would you suggest as the less incredibly conservative approach?
i would have to manually pass these locals around?
jnthn No, not that :) That's horrible :)
timotimo agreed! :)
jnthn The next step isn't too bad actually
So, here's what needs to happen.
timotimo 724.34user 36.68system 7:16.56elapsed 174%CPU (0avgtext+0avgdata 256908maxresident)k
jnthn 1) First, process any nested blocks. 21:55
timotimo except for the 5 minutes of waiting for IO-socket-INET.t to pass, that wasn't too bad )
:)
also, it seems like that test only fails every odd test, rather than all of them %)
jnthn 2) If, after processing a block, all of its lexicals have become locals, and it has blocktype immediate or immediate_static, you can now just inline that block into the place it was enclosed.
timotimo hm, do i have to follow QAST::BVal nodes into these blocks? 21:56
jnthn 3) After doing that for inner ones, now process the curent block. A bunch of lexicals that were out of reach before now may not be.
For immediate blocks, there's no QAST::BVal references to them.
timotimo ah
that would be good indeed
jnthn It's important to only inline blocks marked immediate or immediate_static.
Note that Rakduo is more involved. 21:57
But for NQP this should be OK.
timotimo what kind of fixups are needed to get this inlining happening?
yes, i'm not touching rakudo at all yet :)
(but since rakudo is implemented in nqp, i'm hoping it'll give benefits even there)
jnthn In theory, given $block which is immediate, replace it was a QAST::Stmt node.
timotimo oh! 21:58
jnthn Containing the nodes within the block.
timotimo that seems entirely doable in that case! :)
jnthn In practice...you may hit a tricky problem: overlapping local names.
I don't know. If your lowered names are unique enough, you're probably OK.
timotimo the lowered names look like ltol_originalname123_3241
the whole original name isn't used, because locals with - in them explode on parrot 21:59
jnthn Note that on Parrot, local names can only be \w+ :(
timotimo allows _, too, though
and numbers
jnthn So does \w+ :P
timotimo oh!
right :)
jnthn Is the number on the end unique?
as in, from a .unique(...) call?
timotimo it's the number that gets added by QAST::Node. ... yes
jnthn Yeah, then you should be good. 22:00
timotimo the other number is the lexical depth it was found at
cool
i'll try to sleep. which will inevitably fail at some point in the night and then i'll see if i can start implementing this! :)
jnthn :)
Try to sleep well :) 22:01
timotimo by "if all of its lexicals have become locals", do you mean only vars that were declared in that block that i'm looking to inline or also just mentions of lexical vars?
i'm thinking the latter should be just fine
jnthn Just declared
So you could be able to just look in the first QAST::Stmts of the block, which is where all the declarations live. 22:02
timotimo i'll have to do some extra magic to track lexicals that have previously been "poisoned" that are now no longer poisoned due to inlining 22:03
but other than that it seems kinda doable
should i expect awesome wins after this is done? :)
jnthn Oh, that's why I suggested that you inline first. 22:04
Since then the analysis of what's declared in this block is easier 22:05
timotimo hm, right, i guess i can do the analysis later, bottom-up.
jnthn Yeah, just pass up the posion.
I think you should get some wins out of this, yeah.
Hard to estimate
I suspect that early versions of the specializer will have a much better time with analyzing locals than lexicals also... 22:06
timotimo i'm hoping you'll be pleasantly surprised. and me, too, of course ;)
the specializer refering to moarvm's specializer?
jnthn Yes.
timotimo right. that can do the optimization for rakudo, then 22:07
jnthn 127.0.0.1:4242/moar 22:46
OMG /moar
yays.
tadzik lol que :) 22:47
oh, a server!
jnthn yeay
tadzik \o/
jnthn++
jnthn It jsut takes the thing you GET and then emits <h1>OMG $that_url</h1> :)
tadzik so, first platform with actually useful poll() in 3...2...1.... ;) 22:48
FROGGS *g* 22:49
jnthn++
I'd return <marquee>OMG $that_url</marquee> or <blink>...</blink>, but h1 is cool too :P 22:50
22:55 dalek joined 22:56 PerlJam joined 22:57 masak joined
dalek arVM: 9c4505f | jnthn++ | src/io/syncs (2 files):
Server side sockets.

Seem to work for a basic HTTP server. The IO socket tests don't work out on Windows, but the first one does successfully do the sockets bits, and the issue may be with qqx, used in the test.		 23:28
23:41 cognominal joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!