Thread Pool

I worked across multiple areas of the engine during its development life-cycle, these included the design of the entity component system (ECS), audio and the engines thread pool. In this section I will discuss the thread pool system as that was my sole responsibility, I will cover the components which made up the thread pool and how it was utilised within the Heatstroke engine.

​

Components:​

Thread Safe Queue

This is where the tasks which the worker threads work through are stored. It is required as the thread pool relies on a structure which can be safely accessed by multiple threads without concern of race conditions or invalid data. The thread safe queue is a template class which has members of a mutex, an std :: queue of type T and an std :: condition variable. The mutex is responsible for locking access to the queue so that only a single thread is accessing the data at a given time. The condition variable member is used to notify waiting threads when work is on the queue and ready to be removed. The variable member condition_variable :: notify_one is called after a task is added to the thread safe queue. Once this call has taken place a single worker thread is woken up and locks the class member mutex which allows the thread to access the work on the queue.

​​​​

​Function Wrapper

The function wrappers purpose is to be a common type that can be used to wrap the tasks that are to be added to the task queue. The tasks intended for the work queue are passed into the thread pools submit function, there the task is passed into the constructor of an std :: packaged task object which is pushed to the work queue. The work queue is a queue of type function wrapper, the task is stored in a struct local to the function wrapper class. The () operator is overloaded to call the task passed into the function wrapper class, when tasks are removed from the work queue the task can be called by invoking the () operator. The book C++ Concurrency in Action by Anthony Williams mentions the potential use of std :: function but it cannot store non-copyable objects and the thread pool uses std :: packaged task which is non-copyable object, therefore a custom wrapper class function wrapper is used.​

​

Join Threads

The join threads class purpose is to clean up resources safely in the event of an error. If an exception is thrown before all the threads are joined then the destruction of the join threads class will safely join any joinable threads and prevent further disruption.

​​

​

Functionality:

The thread pool is implemented within its own class, the components discussed above are its member variables. The thread pools member functions are as follows:

​

worker_thread() - This function is where each thread in the thread pool operates. When a thread is created it is assigned to the worker thread function. Inside this function a thread will attempt to remove work from the queue, if there is no work the thread will sleep to limit overhead. The worker thread runs in a loop which runs until the thread pool is destroyed.

​

submit() - Creates a std :: packaged_task object and passes the function for the work queue into this object. The std :: packaged_task object requires the return type of the callable that is passed to it, this is deduced in this function at compile time by std :: invoke_result. A future object is created which is assigned the future from the package task object, this future is returned from this function. When the thread needs to be joined std :: future :: get( ) is called to get the result of the task function that was passed to the work queue.

​

​

Implementation:

Heatstroke's thread pool allows functions with no arguments and return type of void to be added to a task queue. When the thread pool is initialized the system hardware is assessed to see how many threads are available. This is done by querying the number of physical CPU cores using the C++ standard library function std :: thread :: hardware_concurrency(). The returned figure 'n' is the available threads that are then created for the thread pool. These threads that are created are the worker threads.

The worker threads check the task queue for work, if none is available they sleep until they are notified by an std :: condition_variable that work is available for them to take off the queue. This prevents multiple idle threads taking up resources by repeatedly locking a mutex to check the work queue for tasks. This method allows the threads to acquire work without much overhead and means that they don’t miss tasks that need to be completed. The thread pool has the ability for the main thread to wait for tasks that need to be completed before the main thread can continue. This adds flexibility to the scenarios where tasks can be run separately as in the situation where a task is taking a long time to complete the main thread can wait if it depends on the outcome of the operation happening in the secondary thread.

When creating the first person shooter in Heatstroke the thread pool was implemented heavily during initialisation of the game. The initialisation of the game models is run on a separate thread as were the initialisation functions for PhysX and the game audio. While these functions where running, concurrent to the main thread, the user is taken to the main menu. The loading time to start the game depends on how far through the data the individaul threads have got before the user presses start game. Implementing the data loading like this means that the user is not waiting for as long after reaching the main menu of the game.

​

​