Writing code that does async I/O is a total pain in the ass. And I’ve written a whole lot of async I/O code, so I should know.
However, if you’re working in a language with full support for continuations it is a lot easier. JavaScript doesn’t support continuations fully, but it has generators, which is like a lite-version of continuations. That is part of the reason why I’m interested in JavaScript scripting.
So, I’ve been experimenting with how to bind GIO-style async operations in JavaScript so that its easier to write async I/O code. I’ve got something going now that looks pretty ok.
Here is an example that calls an async function “$open” to read three files. It also demonstrates how to do calls from one async generator to another.
do_stuff = new AsyncRunner (function (filenames) {
for (let i in filenames) {
var data = yield $open (filenames[i], 0);
print ("data for " + filenames[i] + " = " + data);
}
var ret = yield do_stuff2.call("other.txt");
print ("AsyncRunner call returned: " + ret);
});
do_stuff2 = new AsyncRunner (function (filename) {
var data = yield $open (filename, 0);
print ("data for " + filename + " = " + data);
yield "the end (of do_stuff2)"
});
do_stuff.start(null, ["test.txt", "test2.txt"]);
Its IMHO pretty easy to understand. At every async op we yield which suspends execution and restarts it at that point when we have the operation results. The “$” in “$open” is just to make it different from the sync variant and still short.
This code is far easier than it would look in C, where we would have to split out a new function at each async function call. The loop in do_stuff makes this quite tricky to do.
The code that implements this is here. Its somewhat complicated, using things like currying, closures and generators, so read it carefully.
I’d like some feedback on this. Are there any tricks I’ve missed that could make this nicer? I really wish javascript had a macro facility so that the yield keyword could be put inside the async calls and we could avoid the boilderplate around the AsyncRunner creation. I would also like to make the AsyncRunner objects callable like “do_stuff2()”, but that doesn’t seem possible with standard javascript (although it is the AsyncRunner class is implemented natively via the spidermonkey APIs…)
Have you tried Ruby for that? I think the same thing could be achieved in a much simpler way.
Funny. Every Stream in .Net has a BeginRead and BeginWrite method. Writing async IO code is about this easy:
{
byte[] buf = new byte[1024];
stream.BeginRead(buf, 0, 1024, (ar) =>
{
/* the read has finished. buf should be full */
buf[0]; /* whatever */
}, null);
/* code here happens before the read is finished, usually. probably just the end of the method. or you could dispatch more IO events. */
}
Oh. Just to educate. I usually combine this with Gtk:
{
byte[] buf = new byte[1024];
stream.BeginRead(buf, 0, 1024, (ar) =>
{
GtkInvoke(() => {
buf[0]; /* can access buffer from in here. on main loop. */
});
}, null);
}
Jerome:
I’m well aware of thar. The GIO async model is stolen from the .net one. However, what you show isn’t significantly easier than in C. Pretty much the only difference is the anonymous function. Try to write the for loop above using that method for instance.
Felipe:
Care to give an example? (Although ruby isn’t a great embedded scripting language for other reasons, see my previous post,)
Interesting, we were just discussing[1] that we should support async method calls in Vala by introducing a yield statement as this would make it very easy to use GIO. We haven’t implemented it yet, though.
[1] http://mail.gnome.org/archives/vala-list/2008-September/msg00053.html
Jürg:
That would be very nice!
What you’re trying to do looks very similar to what Gustavo Carneiro’s GTasklets (http://www.gnome.org/~gjc/gtasklet/gtasklets.html) actually achieve for PyGTK applications. They are not only useful for asynchronous I/O, but for implementing bits and pieces of user interfaces. For example, in one application, I have a fullscreen mode that hides the cursor and a button bar after 10 seconds of mouse inactivity. The whole behavior is cleanly separated in a single tasklet procedure. Normally, this would require some additional attributes, as well as adding code to a few of the methods in the class controlling the fullscreen window, which is usually much less readable and maintainable because the code ends up being spread all over the place.
In a related note, I read your previous post about JavaScript being the quintessencial embedded language, but I really can’t buy into JavaScript as a language for writing serious applications. Even despite Mozilla’s efforts to improve the language in their Spidermonkey implementation (no one else supports yield statements in JavaScript as far as I know) the language has too many flaws to be taken seriously. I could start with variables being global by default (very dangerous!) and end with the lack of a standard class definition mechanism (e.g., everyone rolls out his own, specific, and incompatible class system) but discussing my complete list of pet peeves would require a dedicated website, I guess.
That said, I can believe that Spidermonkey is a lean, speedy interpreter, and that this characteristics make it very attractive for embedding. So bad, this doesn’t make the language it implements any better, though…
Martin:
Interesting. It would be nice to have async sleep and queue_idle availible too, so you could do things like that.
Yield is part of JS 1.7, which should be available in Safari 3.0 too. Maybe in chrome too? I don’t know what JS version V8 implements.
I don’t necessarily disagree with you that JS isn’t the nicest language about, but some of the things you mention can be fixed by standardizing the embedding bindings, and by “grounding” the base part of your app in C/C++. I don’t love JS, but I believe its effective for this kinds of use (as proven by e.g. firefox). I don’t know lua very well, but it seems like a similarly kind of weird language, but still people estimate that about 40% of adobe lightroom is written in that…
“Its IMHO pretty easy to understand”
What!?
What is the add_idle() in the example implementation? Is it part of a JS Library? Thanks
Rosie;
Its a tiny wrapper around g_idle_add(). Its set up in gtk-js.c.
Aaah thanks a lot
The problem with OO in JS is that everyone (okay, most people?) wants to pretend it follows the class/object design of Smalltalk, Python, Java, C++, Eiffel, etc… and then constructs various implementations in an attempt to fit that model.
It isn’t.
It’s a prototype OO system. Like Self or Io.
Leave Java in Java.
(I think this applies to a number of the other “proposals” as well; most seemed fueled by language-comfortably taint.)
Use Javascript as Javascript.
That being said, I’d like a cleanup and -minor expansion- of the language in general: removing object and non-object distinctions, removing ‘var’ in global context, possibly even promoting all local variables to local lexicals by default, making a consistent way to having more control over the prototype, adding a “lightweight” syntax of creating functions (make “blocks” seem more natural), add some form of name-spaces, expand the built-in operations on arrays and other primitive data-types (as standard extensions), create a way to write custom [] and []= methods (and provide alternative always-direct member access), etc.
Javascript really has almost everything that it needs there. Some is buried, some isn’t fully realized and the rest is suppressed.
Interesting article. It would be nice to see the specific Javascript version mentioned though as I’d argue “Javascript” is still ubiquitous with ECMA Script 3 (ECMA-262).
In Ruby you have proper closures, so it’s much easier to do asynchronous stuff:
http://gist.github.com/13492
Also, continuations in C are doable:
http://felipec.wordpress.com/2008/09/28/continuations-in-c-easy-asynchronous-stuff/