github.com/moarvm/moarvm | IRC logs at colabti.org/irclogger/irclogger_logs/moarvm
Set by AlexDaniel on 12 June 2018.
MasterDuke .tell samcv src/strings/decode_stream.c: In function ‘run_decode’: src/strings/decode_stream.c:142:27: warning: implicit declaration of function ‘MVM_string_utf16_decodestream’; did you mean ‘MVM_string_utf8_decodestream’? [-Wimplicit-function-declaration] reached_stopper = MVM_string_utf16_decodestream(tc, ds, stopper_chars, sep_spec, eof); 00:04
yoleaux MasterDuke: I'll pass your message to samcv.
samcv MasterDuke: ah that looks like a thing 00:12
yoleaux 00:04Z <MasterDuke> samcv: src/strings/decode_stream.c: In function ‘run_decode’: src/strings/decode_stream.c:142:27: warning: implicit declaration of function ‘MVM_string_utf16_decodestream’; did you mean ‘MVM_string_utf8_decodestream’? [-Wimplicit-function-declaration] reached_stopper = MVM_string_utf16_decodestream(tc, ds, stopper_chars, sep_spec, eof);
Geth MoarVM: ff0fb63a46 | (Samantha McVey)++ | 2 files
Make sure utf16 decodestream is declared in the header file

Fixes compiler warnings about the function not being predeclared.		 00:29
MasterDuke samcv++ 00:31
samcv and i probably need to check for endianess issues on deing $fh.write: for a 16 bit buffer 00:34
though i need to add specified endianess to decodestream and decode too. so i'll probably do it together with that. since we don't support specific endianess utf16 so i think that's in line with what we have other places atm 00:35
Geth MoarVM/fork-safety: 2587d1bc40 | (Bart Wiegmans)++ | 16 files
[fork] Implement fork() opcode

Stop system threads before fork(), to ensure that the child process is not left with a corrupt heap. Windows will report as fork-not-supported and the process will shortcircuit there.
Some final modifications necessary:
  - let MVM_thread_cleanup_thread_list count the number of alive threads
  - let MVM_io_eventloop_start handle restarts gracefully
  - let MVM_io_eventloop_destroy handle partial stops gracefully		 11:13
nine almost.....there 12:04
nine And it runs! 19:11
jnthn :) 19:12
So much nice stuff going on. :) 19:13
jnthn is looking forward to digging back in this week :)
nine Who'd have thought that the const_i64 -> const_64_16 optimization was essential?
And indeed it compiles and runs a note("hello world") :) 19:15
jnthn Essential? :) 19:16
jnthn didn't know that either
nine Bytecode was kinda broken without that. I got the impression that some upper layer depends on that optimization
jnthn Hmm, that's...odd
nine Or.....oh no! 19:17
Of course it's my own bug :) $!bytecode.write_uint16(nqp::getattr($o, MAST::IVal, '$!value')); # FIXME need to pick int size from op_info
jnthn :D 19:18
jnthn is somewhat relieved :)
nine I even got the infrastructure for fixing that in place since I wrote the comment
Btw the work on reproducible builds now comes in quite handy. Comparing hex dumps is much more fun this way :) 19:22
I guess the next step is plugging this into the compiler backend 19:31
timotimo very nice 19:40
timotimo this way we can even use the debugserver stuff to instrument the compiler, like to mark up every single byte in the resulting file with the line (or maybe a stack) of what wrote it 19:40
timotimo i once did something like this where there was a #define for printf that would put a random color based on $?LINE (and $?COL?) along with a title="..." into an HTML formatted document 19:41
instead of the plain text
Geth MoarVM: bdw++ created pull request #967:
Safe fork in MoarVM		 19:53
nine Meh....seems like there's just no op for loading and running a CompUnit from memory 19:55
timotimo that sucks, we should fix that. 19:55
should be easy, though
but wait, don't we totally compile compunits in memory and run them immediately? 19:56
oh, that's probably the mast compiler doing that for you
nine That's masttocu
timotimo not an nqp op
yeah, we shall have cufrombytes or whatever
brrt nine++ 19:57
nine Instead of nqp::loadbytecodebuffer it'd be better to have an nqp::cufrombuffer + nqp::runcu
Or just have nqp::loadbytecodebuffer return the CompUnit like nqp::masttocu
brrt this should be easy-ish? :-) 19:58
timotimo i wonder how many different ways to use this there are for our compunit repo infrastructure
brrt jnthn: note that this (fork()) only started itching when you wrote the 'sister languages' post 20:00
timotimo that post includes a bit about fork not working?
jnthn :-)
brrt yes
how perl5 is closer to POSIX than perl6 is
timotimo tbqh i like having fork safety
brrt well, I guess that is still true
timotimo preforking can still be useful for resource management reasons
brrt it's limited though; It will only work if there are no user threads active
timotimo oh?
brrt yeah 20:01
timotimo because restoring a stack is hard?
jnthn I think it only sanely *can* work in that case
brrt well, there is that
jnthn Well, there's a caveat to that remark
timotimo so we're basically reproducing the "fork only lets the forking thread survive" semantics 20:01
brrt correct
timotimo but it should be possible to clean the other threads up fully post-fork? 20:02
jnthn timotimo: The patch as it stands throws an exception if there's more than one user thread active
Which I think is sensible
timotimo OK
that's probably a good idea, though with no current API to shut down worker threads in the TPS ...
don't we already get a user thread when we tap a signal for example?
jnthn Yup 20:03
nine brrt++ # awesome work!
jnthn And also if we have to spawn a process to compile a module
brrt I expect it *is* possible to have full thread restore, but that requires another level of hackery that I'm not ready for today
timotimo damn, that's very action-at-a-distance-y
jnthn That's solvable to resolivng what timo said and then making the pre-comp stuff use its own pool
Then we have to consider what happens if the compiler itself were to use threads internally, which is probably desirable at some point too :) 20:04
brrt :-)
nine I'd still much rather pre-comp in the same process. IIRC I've gotten pretty far on that already
timotimo oh, we'll have it in the same process
just with a cleanly separated namespace
that we can just completely tear down with ease
maybe even a complete MVMInstance
jnthn But again, we can probably find a way to deal with that.
brrt note that any of the async stuff will work; so asyncspawn, then fork(), that ought to work
but yeah, a thread pool, that might be difficult.... We probably need to do some HLL logic around that 20:05
jnthn As to supporting it with real userland threads...well...
timotimo i mean, we do offer Thread.join %) 20:06
brrt the perl6 implementation that I'm imagining, will probably have to do something of the sort of $*SCHEDULER.stop(); fork(); $*SCHEDULER.start();
jnthn If we re-invent locks and just use a single per-thread cond var to handle the "efficiently block" part, like the JVM does to back its park() operation, then we could be able to theoretically take some steps further 20:07
But if the thread is in C-land or blocking on I/O and so forth, we're again stuck
timotimo of course, if any thread (worker or otherwise) is blocked, we won't be able to do things right yet
nine well step by step
jnthn And I think that final step is close to unsolvable.
jnthn But 90% solutions are still useful 90% of the time. :) 20:08
timotimo next we implement fork() on windows
jnthn :P
brrt wonders how the linux-subsystem-for-windows does that
timotimo it switches the CPU to no-interrupts-mode for a few seconds :P 20:09
AlexDaniel btw these moar changes can go right in because we're still in pretoasted state 20:15
or so I think
brrt cool 20:16
AlexDaniel and also my bisect logic for toaster almost works, so we'll likely get all the answers right away without needing to wait for me to bisect semimanually 20:18
brrt nice 20:20
AlexDaniel++ # automating things
AlexDaniel well, it doesn't trisect into moar but it will show the right bump right away no matter how many dependencies the module has
also reuses all the logic from bisectable bot so once that is improved to bisect moar it will start doing that as well 20:21
also I'm a bit late with that thing for the release, sorry for that 20:22
samcv ugh utf16 with the BOM handling is really disturbing me 20:26
github.com/MoarVM/MoarVM/issues/96...-421826781 i don't like the ambiguity at all
nine After getting around the obligatory bootstrap issues, the new MAST compiler passes its first tests :) The only oddity is that the test code seems to run twice. 20:29
brrt \o/
Geth MoarVM: 156b5bc3ff | (Stefan Seifert)++ | docs/bytecode.markdown
Clarify a few things in bytecode specification
 20:32
nine Huh? The double run is not because of anything in the bytecode apparently. So it's probably something really stupid in the quickly hacked up nqp::runbytecodebuffer or so 20:44
timotimo if the code is in the mainline, then loading it probably already runs it? 20:46
and then you run it again? 20:47
nine Ah, of course. mast_to_cu does not run the whole thing. It just runs the deserialization frame. So indeed, just hacking up loadbytecodebuffer to return the CU won't do it. 20:48
But at least I now know that when I solve this for real, the rest is already in pretty good shape :)
nine And indeed it is. And that concludes my hacking for this weekend 21:18
jnthn nine++ 21:19
timotimo way cool. 22:10

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!