Async Rust: history strikes back.

Those who do not study history, are doomed to repeat it.

Those who study history, are doomed to sit and watch while some idiot repeats it.

Last week we have released a new version of Glommio, a thread-per-core asynchronous executor for Rust. Having each individual executor working within the confines of a single thread allow us to make a lot of guarantees that the ecosystem at large can’t. In our latest release, we were positive we had crafted a well defined API that would simplify the creation of asynchronous Glommio programs. Only to find out we were simply repeating the mistakes of giants that came before us.

As a rule of thumb, we as people like to talk about our successes and not so much about our failures. Upending that a bit, I figured I would take the time to write about that recent failure, reflect a bit about how asynchronous Rust can be much harder and full of surprises than it seems, and what this means for its future.

The problem

One of the well-known sources of friction for newcomers in Rust is the borrow checker with its lifetime rules. Yet, the borrow checker is what gives Rust its unique flavour and guarantees memory safety: the compiler enforces those rules for you, and as long as you don’t write code marked with the unsafe keyword, memory corruption is impossible.

Despite its initial friction, lifetimes are, once understood, usually easy to handle: an object will be alive until it goes out of scope, and you can only keep one mutable reference to it at a time.

This model gets a bit more complex with asynchronous code: by its very nature, asynchronous code can execute at any time in the future (or not at all), which makes scoping unpredictable. Because of that, code like this requires a'static bound: a special lifetime parameter that indicates that an object is alive for the entire duration of the program.

See for example, how to spawn a new thread in Rust: