How the JVM Thread Pool Scales Up

In the HotSpot VM threading model, Java threads map one-to-one to operating system threads (virtual threads arrived in JDK 19, but traditional threads still follow this model). In other words, every Java thread is backed by an OS kernel thread (KLT / LWP).

When Java creates a thread, it has to call a kernel API to create the corresponding kernel thread, and the OS must allocate stack space, scheduling metadata, and other resources for it. When the Java thread terminates, the kernel thread is reclaimed as well. As a result, thread creation and destruction are expensive, and the number of threads cannot grow without bound.

When there are too many threads:

1. Creating them incurs significant system overhead
2. The CPU spends time constantly performing context switches between threads
3. A large number of threads can degrade system performance or even cause an OOM

That is why, in practice, we use a thread pool (ThreadPoolExecutor) to manage thread lifecycles centrally.

Core ThreadPoolExecutor Parameters

Parameter	Type	Description
corePoolSize	int	Core pool size — the number of threads kept alive in the pool long-term
maximumPoolSize	int	Maximum pool size — the upper bound on threads that may be created once the task queue is full
keepAliveTime	long	Maximum idle time for non-core threads before they are terminated
unit	TimeUnit	Time unit for keepAliveTime
workQueue	BlockingQueue	Task queue holding tasks waiting to be executed
threadFactory	ThreadFactory	Thread factory, used to customize thread names, priorities, and so on
handler	RejectedExecutionHandler	Rejection policy — what to do when a task cannot be executed

How the Pool Handles Tasks

When a new task is submitted, the pool processes it in the following order.

1) Core threads run the task first

If the current thread count is less than corePoolSize, the pool creates a new thread right away to run the task.

2) Core threads full → task goes into the queue

If the thread count has already reached corePoolSize, a new task does not immediately trigger thread creation; instead it goes into the BlockingQueue task queue and waits.

3) Queue full → pool scales up

If the task queue is at capacity and the current thread count is less than maximumPoolSize, the pool creates new threads to handle tasks. These are called non-core threads.

4) Beyond the maximum → rejection policy

If the queue is full and the thread count has reached maximumPoolSize, the pool triggers the RejectedExecutionHandler rejection policy.

Common policies:

• AbortPolicy (the default — throws an exception)
• CallerRunsPolicy
• DiscardPolicy
• DiscardOldestPolicy

What keepAliveTime Does

When a non-core thread in the pool executes no tasks within keepAliveTime, it gets destroyed. Core threads are not destroyed by default, while non-core threads idle beyond keepAliveTime are reclaimed. This lets the pool scale up under peak load and shed threads when load is low.

A Pitfall That Is Very Easy to Hit

If workQueue is an unbounded queue (e.g. LinkedBlockingQueue with its default constructor), tasks keep flowing into the queue and it practically never fills up. The pool therefore never reaches the scale-up step, and maximumPoolSize and keepAliveTime essentially never take effect. This is a common reason why thread pool configuration silently does nothing in many production systems.

Summary

The pool's scaling path boils down to: core threads → task queue → non-core threads → rejection policy. maximumPoolSize only kicks in after the queue is full, and keepAliveTime determines when non-core threads are reclaimed. When configuring a pool, the choice of queue type matters just as much as the thread count parameters — an unbounded queue renders both scaling and the rejection policy meaningless. Once you understand this full chain, most production "my thread pool settings aren't working" mysteries have an obvious answer.