Threads don’t scale? They might do now!
One of the interesting things in this industry is that every once in a while you have to completely re-evaluate your beliefs in light of new developments. It’s long been common sense that thread-based servers scale worse than selector-based poll/NIO ones.
This certainly was true at one point in time, but it seems it may not be true any longer. There’s several reasons:
- Modern threading libraries seem to have greatly increased the number of threads they can run.
- Locking/synchronization is now orders of magnitude faster and can sometimes even be optimized away by the compiler.
- Threading libraries are much better at context switching sometimes eliminating them completely (why switch in a thread I know is waiting for I/O?) and having constant time schedulers (context switching doesn’t get more expensive the more threads you have).
It’s funny that while the whole industry was learning how to do designs for non-threaded models there was this group of people fixing up the actual infrastructure itself.
Time to pick up Doug Lea’s threading book again. Maybe this reading I will understand some of it.
March 14th, 2008 at 4:45 am
I read this post too!
As we were talking about the other day (over beers =), select/poll i/o for threads is about more than scaling. In a pre-forking setup (synchronous thread-per request), a clever DoSer could open slow connections and eat up the maximum thread limit without putting any real computational load on the CPU(s).
The more threads you open the more expensive the context management and held resources can. Even though switches may be faster now, switching hundreds or thousands of zombied threads is itself a drag in this kind of setup.
Note that java.nio’s async I/O is also highly dependent on the OS and machine impl of select/poll. Mac OS has a weaker impl of this than Solaris for e.g. (atleast it used to sometime ago).
I wouldn’t be so quick to write off select/poll-driven non-blocking web servers =)