Nbsp;   How the Thread Pool Manages Its Threads

Now I’d like to talk about how the thread pool code manages worker and I/O threads. However, I don’t want to go into a lot of detail, because the internal implementation has changed greatly over the years with each version of the CLR, and it will continue changing with future versions. It is best to think of the thread pool as a black box. The black box is not perfect for any one application, because it is a general purpose thread-scheduling technology designed to work with a large myriad of appli- cations; it will work better for some applications than for others. It works very well today, and I highly recommend that you trust it, because it would be very hard for you to produce a thread pool that works better than the one shipping in the CLR. And, over time, most applications should improve as the thread pool code internally changes how it manages threads.

Setting Thread Pool Limits

The CLR allows developers to set a maximum number of threads that the thread pool will create. However, it turns out that thread pools should never place an upper limit on the number of threads in the pool because starvation or deadlock might occur. Imagine queuing 1,000 work items that all block on an event that is signaled by the 1,001st item. If you’ve set a maximum of 1,000 threads, the 1,001st work item won’t be executed, and all 1,000 threads will be blocked forever, forcing end users

to terminate the application and lose all their work. Also, it is very unusual for developers to artificially limit the resources that they have available to their application. For example, would you ever start your application and tell the system you’d like to restrict the amount of memory that the application can use or limit the amount of network bandwidth that your application can use? Yet, for some rea- son, developers feel compelled to limit the number of threads that the thread pool can have.

Because customers have had starvation and deadlock issues, the CLR team has steadily increased the default maximum number of threads that the thread pool can have. The default maximum is now about 1,000 threads, which is effectively limitless because a 32-bit process has at most 2 GB of usable address space within it. After a bunch of Win32 DLLs load, the CLR DLLs load, the native heap and the managed heap is allocated, there is approximately 1.5 GB of address space left over. Because each thread requires more than 1 MB of memory for its user-mode stack and thread environment block (TEB), the most threads you can get in a 32-bit process is about 1,360. Attempting to create more threads than this will result in an OutOfMemoryException being thrown. Of course, a 64-bit process offers 8 terabytes of address space, so you could theoretically create hundreds of thousands of threads. But allocating anywhere near this number of threads is really just a waste of resources, especially when the ideal number of threads to have is equal to the number of CPUs in the machine. What the CLR team should do is remove the limits entirely, but they can’t do this now because doing so might break some applications that expect thread pool limits to exist.

The System.Threading.ThreadPool class offers several static methods that you can call to manipulate the number of threads in the thread pool: GetMaxThreads, SetMaxThreads, GetMin­ Threads, SetMinThreads, and GetAvailableThreads. I highly recommend that you do not call any of these methods. Playing with thread pool limits usually results in making an application perform worse, not better. If you think that your application needs hundreds or thousands of threads, there

is something seriously wrong with the architecture of your application and the way that it’s using threads. This chapter and Chapter 28 demonstrate the proper way to use threads.

How Worker Threads Are Managed

Figure 27-1 shows the various data structures that make up the worker threads' part of the thread pool. The ThreadPool.QueueUserWorkItem method and the Timer class always queue work items to the global queue. Worker threads pull items from this queue using a first-in-first-out (FIFO)

algorithm and process them. Because multiple worker threads can be removing items from the global queue simultaneously, all worker threads contend on a thread synchronization lock to ensure that two or more threads don’t take the same work item. This thread synchronization lock can become a bottleneck in some applications, thereby limiting scalability and performance to some degree.

FIGURE 27-1The CLR's thread pool.

Now let’s talk about Task objects scheduled using the default TaskScheduler (obtained by querying TaskScheduler’s static Default property).4 When a non-worker thread schedules a Task, the Task is added to the global queue. But, each worker thread has its own local queue, and when a worker thread schedules a Task, the Task is added to calling the thread’s local queue.

When a worker thread is ready to process an item, it always checks its local queue for a Task first.

If a Task exists, the worker thread removes the Task from its local queue and processes the item. Note that a worker thread pulls tasks from its local queue by using a last-in-first-out (LIFO) algorithm. Because a worker thread is the only thread allowed to access the head of its own local queue, no thread synchronization lock is required and adding and removing Tasks from the queue is very fast. A side effect of this behavior is that Tasks are executed in the reverse order that they were queued.

If a worker thread sees that its local queue is empty, then the worker thread will attempt to steal a Task from another worker thread’s local queue. Tasks are stolen from the tail of a local queue and require that a thread synchronization lock be taken, which hurts performance a little bit. Of course, the hope is that stealing rarely occurs, so this lock is taken rarely. If all the local queues are empty, then the worker thread will extract an item from the global queue (taking its lock) using the FIFO algorithm. If the global queue is empty, then the worker thread puts itself to sleep waiting for some- thing to show up. If it sleeps for a long time, then it will wake itself up and destroy itself, allowing the system to reclaim the resources (kernel object, stacks, TEB) that were used by the thread.

4 Other TaskScheduler-derived objects may exhibit behavior different from what I describe here.

The thread pool will quickly create worker threads so that the number of worker threads is equal to the value pass to ThreadPool’s SetMinThreads method. If you never call this method (and it’s recommended that you never call this method), then the default value is equal to the number of CPUs

that your process is allowed to use as determined by your process’s affinity mask. Usually your process

is allowed to use all the CPUs on the machine, so the thread pool will quickly create worker threads up to the number of CPUs on the machine. After this many threads have been created, the thread pool monitors the completion rate of work items and if items are taking a long time to complete (the meaning of which is not documented), it creates more worker threads. If items start completing quickly, then worker threads will be destroyed.

C HA P T E R 2 8

Date: 2016-03-03; view: 817