Turning Two Decades into Rust

We have embarked on a new journey to modernize our server technologies. For the last twenty years or so, the database server has been written in Delphi, a technology that has served us very well. However, as the technological landscape has evolved, we have recognized the need to explore alternatives. Our goal is to find a replacement that is more popular, still performant, but also a language that developers want to use. Our conclusion was Rust.

Rust is a systems programming language that has been the most loved/desired language for the last 8 years of Stack Overflow Developer Surveys. With its unique memory management system, it can make guarantees that basically no other languages can. It is known for performance, type safety, and its vibrant community. It comes with a complete set of tooling, including a build system and a package manager, which makes it extra convenient to develop your own or someone else’s package. In fact, the number of packages being developed for Rust is currently doubling each year with over 144 million being downloaded each day.

The heavy hitters

We are not the first company to rewrite something in Rust. One of the first major applications of Rust after its inception was in rewriting components of Mozilla’s Firefox browser. The same company whose research team birthed Rust. Since then, others have followed suit, including Microsoft, Figma, and Dropbox. This is just a small selection of companies because it would be hard to find any major tech company that is not at least exploring the language. Some take it easy, like Netflix, and write just a couple of tools, while others like Android write 1.5 million lines of code or Cloudflare who wrote a framework to serve 1 trillion requests per day and ended up serving 1 quadrillion requests after 1 ½ years with no downtime.

These gigantic companies not only put a lot of trust in this relatively young technology, but they also support the community of maintainers by donating millions of dollars to the Rust Foundation, the non-profit organization that distributes resources to developers to enhance the ecosystem. The foundation makes sure no big entity takes sole control and everyone can benefit from everything.

We have a plan

At this early stage, we have no doubts we bet on the right technology. The bigger worry is the resources that are being locked up during the rewrite, and whether it creates more problems than it solves. Everyone who makes this journey should ask themselves, will it be worth it, how much progress could have been made in the same amount of time as the rewrite? For us, it is an investment for the future, in technology and talent acquisition.

We have made two recruits so far to work solely on the server rewrite. In addition, the author of the original Delphi server and two other employees have also been assigned to work part-time on the reimplementation. The time horizon is an initial 4-month explorative proof of concept (PoC) phase where we are iterating over smaller subproblems and building a slice of functionally, followed by a year of implementation. The result of the PoC is also the starting point for the implementation itself. When it comes to our complete software stack, this server amounts to less than 10% of all code.

To increase the probability of success, we are taking a conservative approach. We believe adding new functionality while migrating the server is a recipe for disaster. Instead, we are making a drop-in replacement that can be tested in the same way as the original server. It might be tempting to try to improve things on a fundamental level along the way. But if we resist, we can take advantage of the original author’s wisdom and avoid reinventing bugs.

And this is just the start

In conclusion, our journey towards adopting Rust is not just a technological upgrade but a strategic move towards future-proofing our infrastructure. By embracing Rust, we are not only addressing the immediate challenge of finding developers but also positioning ourselves at the forefront of modern software development. Our commitment to a conservative approach in server migration ensures that we mitigate risks while setting a solid foundation for growth and innovation. The rewrite may temporarily divert resources, but we are confident that the long-term benefits will far outweigh the initial investment.

We look forward to sharing our experiences and the lessons learned at a later stage. We believe that our journey will not only lay a solid foundation for many future enhancements to our server’s capabilities but also inspire others to consider the potential of Rust in their technological endeavors.