Rust for JVM Engineers: Memory Safety Without a Garbage Collector

#Distributed Systems 10 min read
Rust for JVM engineers: an arch bridge with no safety net below, memory safety held by ownership, not a garbage collector.

For most of my career, memory was something I delegated.

On the JVM you allocate freely and let the garbage collector clean up behind you. You never call free. You rarely think about when an object dies. The runtime decides, and most of the time the runtime decides well enough that you never look. That is the deal the JVM offers, and for a generation of backend services it has been a good deal.

The deal has a clause most of us read only when it bites: you do not control when the collector runs.

Rust takes the opposite position. In safe Rust it gives you memory safety, no use-after-free, no double-free, no data races on shared memory, without a garbage collector at all. There is also an unsafe Rust, where you can opt out of some of these checks for the cases the compiler cannot prove, and the guarantees then become yours to uphold. But the default, the part you write almost all the time, is the safe subset. Coming from Scala, that combination did not sound possible to me at first. Safety in the managed world has always meant a runtime watching your back. Safe Rust claims the same safety with no runtime watcher. The interesting question is not whether that is true. It is true. The interesting question is what it actually buys a backend engineer who already has a perfectly good GC.

Safety in a managed language is a property the runtime enforces. In safe Rust it is a property the compiler proves.

That sentence is the whole shift, and the rest follows from it.

What the garbage collector actually costs

The JVM's garbage collector is one of the most refined pieces of systems software in existence. Decades of work went into G1, then ZGC and Shenandoah, specifically to shrink the pauses. This is not a story about a bad collector. It is a story about a real and irreducible tradeoff.

A garbage collector trades throughput and determinism for convenience. It runs on its own schedule, not yours. It scans, it moves, it reclaims, and while it does the most aggressive of that work, your application threads are not the ones deciding what happens next. Modern collectors have pushed pause times down impressively, but "down" is not "gone," and the timing is not yours to set.

For the average service this is invisible. Your p50 latency does not care about the collector. Your throughput is fine. The GC earns its keep by letting you ship features instead of managing arenas.

The cost shows up in the tail.

It is the p99 and the p999, the request that happened to land during a collection cycle, the one user in a thousand whose call took ten times as long for no reason visible in the code. In a service where the tail is the product, a trading gateway, a low-latency API, an ad auction with a hard deadline, a streaming data plane, that tail is not a footnote. It is the thing you are paid to control. And it is precisely the thing a garbage collector reserves the right to disturb.

The collector also costs memory headroom. Generational, tracing collectors want slack to work efficiently; run them close to the heap ceiling and they collect more often and pause harder. You pay for the convenience in RAM you provision but do not get to use for data.

None of this makes the JVM wrong. It makes the JVM a particular choice with a particular bill, and most of us pay that bill without ever reading it.

Ownership is the idea, not the syntax

Rust's answer is ownership, and ownership is an architectural idea before it is a language feature.

For an ordinary owned value, there is exactly one owner. When the owner goes out of scope, the value is freed, deterministically, at a point you can see in the code. There is no collector deciding later. The deallocation happens at the closing brace, in the same place. That is the model for the common case. Shared ownership through Rc and Arc, and deliberate leaks through mem::forget, complicate the simple picture, but they are the exceptions you reach for on purpose, not the default. C++ programmers will recognize the underlying idea as RAII, and that lineage is exactly right. Rust took deterministic destruction and made the compiler enforce the rules that make it safe.

Around ownership sits borrowing. You can lend a value out by reference, and the borrow checker enforces a single discipline at compile time: you may have many readers, or one writer, but never both at once. That one rule, aliasing xor mutability, is the quiet center of the whole language. It is also, not coincidentally, the rule that prevents data races in safe Rust. A data race requires two threads touching the same memory with at least one writing. In the safe subset the borrow checker makes that arrangement fail to compile.

This is why Rust people talk about "fearless concurrency," and the phrase is more precise than it sounds. It does not mean concurrency becomes easy. It means a specific and vicious class of concurrency bug, the shared-mutable-state data race, is moved from runtime, where it is nearly impossible to reproduce, to compile time, where it is just an error message. For anyone who has spent a week chasing a heisenbug that vanishes under a debugger, moving that bug class to the compiler is worth a great deal.

The thing to understand, coming from the JVM, is that none of this is paid for at runtime. There is no tracing, no scanning, no background thread. The cost is paid once, at compile time, by you, in the form of a compiler that refuses programs the borrow checker cannot prove safe. You are trading a runtime tax you cannot schedule for a compile-time tax you pay up front.

What this buys a backend engineer

Strip away the language enthusiasm and the architectural value is narrow and real.

You get predictable resource use. Memory is freed at known points, so the memory profile of a Rust service tends to be flat and legible rather than a sawtooth of allocation and collection. There is no heap headroom reserved for a collector to breathe. The footprint is closer to what the data actually needs.

You get latency without a collector in the path. The tail is no longer subject to a pause you did not schedule. This is exactly the motivation behind public migrations like Discord's move of a latency-sensitive service from Go to Rust in 2020: a Go service was fine on average and bad on the tail because of garbage-collection spikes, and Rust removed the collector from the equation entirely. The lesson generalizes past the specific languages. When the tail is the product, the collector is in your critical path whether you scheduled it or not.

You get deterministic destruction, which is more general than memory. Because cleanup happens at a known point when the owner drops, the same mechanism that frees memory also closes the file, releases the lock, returns the connection to the pool, at a place you can see. Resource lifetime stops being a thing you hope finalizers eventually handle and becomes a thing the type system guarantees.

That is the buy. Not magic, not universal, but specific and defensible: predictable resource use and a tail latency that belongs to you, achieved without surrendering memory safety.

The honest tradeoffs

I am not going to pretend the trade is free, because it is not, and the cost is exactly where the benefit is.

Ownership has a real learning cost. The borrow checker that prevents your data races will also, for the first weeks, reject programs you are certain are correct, because you have not yet internalized the aliasing-xor-mutability discipline. The famous "fighting the borrow checker" phase is real. What is happening is that the compiler is forcing you to make ownership decisions explicit that a GC let you leave implicit. That clarity is the point, and it is also friction, and on a team it is a hiring and ramp cost you have to budget honestly.

The ecosystem trade is real too. The JVM is thirty years deep. The libraries, the profilers, the operational tooling, the institutional knowledge of how a JVM service behaves at 3am, all of that is mature in a way few ecosystems match. Rust's ecosystem is strong and growing, but for a large class of business services the JVM is simply the lower-risk, faster-to-ship choice, and that is a legitimate architectural verdict, not a failure of nerve.

And the garbage collector is genuinely the right call for most services. If your tail latency is not the product, if your team's throughput matters more than your service's p999, if the domain is a CRUD-shaped business application, the GC is doing you a favor by taking memory off your plate. Reaching for manual ownership there is paying the borrow checker's tax to solve a problem you do not have. The discipline is to know which problem you actually have.

The takeaway

What Rust changed for me was not a language preference. It was an assumption.

I had quietly believed that memory safety required a runtime to enforce it, and therefore that the garbage collector's bill was the price of safety. Rust separates those two things. Safety can be a compile-time proof rather than a runtime service, and once it is, the collector becomes optional rather than load-bearing. You keep it when its convenience is worth its tail, and you put it down when the tail is the thing you are paid to protect.

The JVM is not displaced by this. It remains the correct default for an enormous range of work, and I would still reach for it without apology on most services. But the choice is now a choice, made with eyes open, rather than a default I never noticed I was accepting.

The garbage collector is not free. It is convenient. Knowing the difference is what makes it a decision.

Sources