Virtual Threads in Java: Project Loom Becomes Reality

There are features the Java community has been waiting for for years. Virtual Threads are undoubtedly one of them. Since September 2022, they have been available as a preview feature in Java 19 (JEP 425), and Java 20 in March 2023 already brought the second preview iteration (JEP 436). After years of development under the project name "Loom," the promise of lightweight concurrency in Java is finally becoming tangible.

At encircle360, we have been trying out Virtual Threads in internal prototypes since the Java 19 release and would like to share our findings. What is behind them, when do they help -- and where are the limits?

The Problem: Thread-per-Request Hits Its Limits

The classic model for server applications in Java is simple: for each incoming request, a thread is assigned that processes the request synchronously. In Spring Boot, Jakarta EE, or any servlet container, this is the standard. A request comes in, a thread from the pool takes it over, processes it, and returns it.

This works well -- until it doesn't. A platform thread in Java is a thin wrapper around an operating system thread. And OS threads are expensive: each typically consumes 1 MB of stack memory, creating and destroying them is costly, and context switches between thousands of threads put significant strain on the kernel scheduler. In practice, this means: a typical server handles a few thousand concurrent threads before memory or scheduling costs become the bottleneck.

For years, this was not a problem because most applications did not need thousands of concurrent connections. But with microservices calling each other and database queries where the thread spends 90% of its time waiting, the thread limit becomes the throughput bottleneck. The application is not CPU-bound -- it is just waiting on I/O, but the threads run out.

Previous Solutions: Reactive and the Cost of Complexity

The Java world's answer to this problem was reactive programming: Project Reactor, RxJava, Spring WebFlux. Instead of blocking one thread per request, you work with non-blocking callbacks and event loops. A few threads serve many requests.

This works technically very well. But it comes at a high price: all code must be written in the reactive style. Sequential methods become Mono and Flux chains. Stack traces become unreadable. Debugging becomes detective work. And every library in the stack must be reactively compatible -- a single blocking call in the chain can bring the entire system to a halt.

In our Spring Boot 3 article, we had already raised the question "WebFlux or classic stack?" Virtual Threads now offer a third way: keep the familiar, synchronous programming style, but with lightweight threads that allow blocking without wasting resources.

What Are Virtual Threads?

Virtual Threads are threads managed by the JVM -- not by the operating system. They use the same java.lang.Thread API that Java developers have known since 1996. Code that works with platform threads also works with Virtual Threads. The crucial difference lies under the hood.

When a Virtual Thread encounters a blocking call -- a database query, an HTTP request, a Thread.sleep() -- it is not held on the OS thread. Instead, it is unmounted from its carrier thread, and the carrier thread can immediately execute another Virtual Thread. Once the blocking operation completes, the Virtual Thread is mounted onto an available carrier thread and resumes its work.

The carrier threads are a pool of platform threads -- by default a ForkJoinPool whose size matches the number of CPU cores. A server with 8 cores thus typically uses 8 carrier threads, which can, however, serve hundreds of thousands or even millions of Virtual Threads.

The memory overhead of a Virtual Thread is just a few kilobytes rather than the typical megabyte of a platform thread. Creation and destruction are nearly free. This fundamentally changes the assumptions for concurrency in Java.

Virtual Threads in Practice

Creating a Virtual Thread is remarkably simple:

// Start a single Virtual Thread
Thread.startVirtualThread(() -> {
    System.out.println("Hello from a Virtual Thread!");
    System.out.println("Thread: " + Thread.currentThread());
});

// Or via the builder
Thread vThread = Thread.ofVirtual()
        .name("my-virtual-thread")
        .start(() -> doSomeWork());

For server applications, the new Executor is the more practically relevant API:

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    // Each task gets its own Virtual Thread
    for (int i = 0; i < 100_000; i++) {
        executor.submit(() -> {
            // Simulated I/O call
            Thread.sleep(Duration.ofSeconds(1));
            return fetchDataFromDatabase();
        });
    }
}
// The try-with-resources block waits until all tasks are complete

This example creates 100,000 Virtual Threads. Each sleeps for one second and then executes a database call. With platform threads, this would be unthinkable -- 100,000 OS threads would require 100 GB of memory for stacks alone. With Virtual Threads, this might occupy a few hundred megabytes.

For comparison, the same approach with platform threads:

// Platform Threads -- only works with a small pool
try (var executor = Executors.newFixedThreadPool(200)) {
    for (int i = 0; i < 100_000; i++) {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return fetchDataFromDatabase();
        });
    }
}
// 100,000 tasks through 200 threads: ~500 seconds instead of ~1 second

With 200 platform threads, processing 100,000 tasks that each block for one second takes around 500 seconds. With Virtual Threads -- where each task gets its own thread and blocking waits free up the carrier thread -- it takes just a few seconds. Throughput for I/O-heavy workloads increases dramatically.

When Virtual Threads Help -- and When They Don't

Virtual Threads are not a silver bullet. They shine with I/O-bound workloads: HTTP requests, database queries, file access, message queue interactions. Wherever threads spend most of their time waiting, Virtual Threads can multiply throughput because the carrier threads are never idle.

For CPU-bound workloads, Virtual Threads offer no advantage. When a thread is computing continuously, there is nothing to unmount. A CPU-intensive algorithm runs just as fast (or slow) on a Virtual Thread as on a platform thread. The number of concurrently computing threads is limited by CPU cores regardless.

Pitfalls and Limitations

Pinning with synchronized

The currently most important limitation: when a Virtual Thread blocks inside a synchronized block, it cannot be unmounted from its carrier thread. It is "pinned" -- the carrier thread is blocked and cannot serve other Virtual Threads. This undermines the entire advantage.

The solution is to replace synchronized with ReentrantLock:

// Problematic: Virtual Thread gets pinned
synchronized (lock) {
    resultSet = statement.executeQuery(); // Blocks -- carrier is pinned
}

// Better: ReentrantLock allows unmounting
private final ReentrantLock lock = new ReentrantLock();

lock.lock();
try {
    resultSet = statement.executeQuery(); // Blocks -- carrier is released
} finally {
    lock.unlock();
}

This affects not only your own code but also libraries. JDBC drivers, connection pools, and other infrastructure libraries often use synchronized internally. The ecosystem needs to catch up here -- and it is already doing so.

ThreadLocal Variables

ThreadLocal works with Virtual Threads, but the semantics change. With platform threads, you have a few hundred threads, and ThreadLocal memory is manageable. With millions of Virtual Threads, memory consumption from ThreadLocals can explode. The JDK team is working on Scoped Values (JEP 429) as an alternative, though these are also still in preview status.

Thread Pools Become an Anti-Pattern

A subtle but important point: thread pools no longer make sense with Virtual Threads. A pool exists to reuse expensive resources (OS threads). Virtual Threads are so cheap that you should create a new one for every task. Executors.newVirtualThreadPerTaskExecutor() is deliberately named this way -- one thread per task, no pool.

Virtual Threads vs. Reactive Programming

The question naturally arises: do we still need Spring WebFlux if we have Virtual Threads? The answer is nuanced.

Virtual Threads solve the same fundamental problem -- efficiently handling I/O wait -- but with the familiar synchronous programming model. For the vast majority of applications that would currently need to be written reactively to be scalable, Virtual Threads will be the simpler and more maintainable solution.

Reactive programming still has its place, however: backpressure, stream processing, and the reactive programming model itself offer advantages that go beyond pure thread efficiency. Those who need reactive streams will continue to use Reactor or RxJava. But anyone who has been programming reactively only because thread pools were not large enough now gets a much more pleasant alternative with Virtual Threads.

Preview Means: Not Yet for Production

An important note: Virtual Threads are a preview feature in Java 19 and 20. This means you must explicitly activate them with --enable-preview, and Oracle reserves the right to change the API between preview iterations. For production systems, we therefore recommend waiting.

The good news: Java 21 is scheduled for September 2023 as the next LTS release, and everything indicates that Virtual Threads will reach their final status there. That would be a momentous step -- the first LTS release with Virtual Threads as a full-fledged feature. Combined with the innovations we presented in our Java 17 article -- Sealed Classes, Records, Pattern Matching -- Java 21 will be a release that fundamentally modernizes the language.

How We Are Preparing

At encircle360, we are already preparing for the transition:

1. Replace synchronized with ReentrantLock where blocking I/O operations occur within critical sections
2. Review ThreadLocal usage and reduce where possible
3. Evaluate library compatibility -- especially JDBC drivers and connection pools
4. Build prototypes with --enable-preview on Java 20 to get a feel for behavior under load
5. Plan for Java 21 LTS in September as the target platform for production use

Virtual Threads do not change how we write Java code -- they change how the JVM executes it. And that is precisely where their elegance lies: existing synchronous code becomes more scalable without a single line needing to be changed. We are looking forward to September.