I have spent almost the entire last decade in a fairly specialized product company, building high performance I/O systems. I had the opportunity to see storage technology evolve rapidly and decisively. Talking about storage and its developments felt like preaching to the choir.

This year, I have switched jobs. Being at a larger company with engineers from multiple backgrounds I was taken by surprise by the fact that although every one of my peers is certainly extremely bright, most of them carried misconceptions about how to best exploit the performance of modern storage technology leading to suboptimal designs, even if…

Recently, I have been contacted by a number of people asking for career advice. Questions usually range from the domain-specific, like how to become successful in low-level systems programming, to the more general like how to reach the level of Staff Engineer.

I was very pleased with that, as it gave me the opportunity to have deep and meaningful conversations, and also look back at my own career and reflect on the subject myself. …

I have recently written about how major changes in storage technology are changing conventional knowledge on how to deal with storage I/O. The central thesis of the article was simple: As fast NVMe devices become common place, the impact of the software layer gets bigger. Old ideas and APIs, designed for a time in which storage accesses were in the hundreds of milliseconds should be revisited.

In particular, I investigated the idea that Buffered I/O, where the operating system caches data pages on behalf of the user should always be better than Direct I/O, where no such caching happens. Once…

Those who do not study history, are doomed to repeat it.

Those who study history, are doomed to sit and watch while some idiot repeats it.

Last week we have released a new version of Glommio, a thread-per-core asynchronous executor for Rust. Having each individual executor working within the confines of a single thread allow us to make a lot of guarantees that the ecosystem at large can’t. In our latest release, we were positive we had crafted a well defined API that would simplify the creation of asynchronous Glommio programs. …

Edit: due to a trademark issue, the project formerly known as Scipio was renamed to “Glommio”. The article was edited to match.

I have recently released a new Rust library aimed at easing the task of writing asynchronous Thread-per-Core applications: Glommio. I intend to use it to power the new generation of storage intensive systems I am writing for my current employer, Datadog.

But I am no novice to such systems: for the past 7+ years I have worked for ScyllaDB, a NoSQL database that managed to consistently post 5 to 10x performance improvements over its rivals, in large part…

Linux is around for almost three decades by now. In its early days contributions were managed manually by Linus himself without any versioning system whatsoever. In modern times, by using git.

Throughout all of that time, however, one thing has been constant: code is sent to one (or many) mailing lists, where it reviewed and debated until it is deemed ready for inclusion.

But despite its success, this process has been coming under fire. An article by Microsoft’s Sarah Novotny recently made quite the splash on social media by claiming that the collaboration tools used by Linux are outdated and…

Glauber Costa

Veteran infrastructure engineer with decades of experience in low-level systems. Previously Linux Kernel and ScyllaDB. Now at Datadog.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store