System Architecture

Fig 1. Level 0 System Architecture

The system was developed using C++ and DirectX 9.0c. DirectX 9 was chosen over the later version of DirectX 10 due to more familiarity of this particular version of the SDK and also because the support for 3d models in the X file format has been withdrawn in the later version in favour of the SDKMesh file format. The system utilises the OpenCV library. OpenCV is a computer vision library originally developed by Intel as part of their initiative to advance CPU-intensive applications. It is free for use under the open source BSD license. The library is cross-platform. It focuses mainly on real-time image processing. The library is mainly written in C, which makes it portable to some specific platforms. However, since version 2.0, OpenCV includes both its traditional C interface as well as a new C++ interface that seeks to reduce common programming errors when using OpenCV in C. Much of the new developments and algorithms in OpenCV are in the C++ interface. Using OpenCV is advantageous as it gives you a valuable head start in developing your own vision application instead of having to reinvent the basic functions from scratch so allowing you to build on top of what came before so enabling ambitious projects to be developed.

As any form of image processing is notorious for being CPU-intensive and the fact that the aim was to try and achieve this in real-time, it was decided early on to try and maximise the performance of the code. The biggest performance enhancement that can be made is to make the application multithreaded, particularly with the image processing running in separate parallel threads to the rendering to the screen. Threads can simplify program design and implementation and also improve performance, but thread usage requires care to ensure that shared resources are protected against simultaneous modification and that threads run only when appropriate. This is particular pertinent within this system as there will be a large number of threads relying on the image data produced by the cameras. Therefore the code needs to be made thread-safe, i.e. several threads can execute the code simultaneously without any undesirable results. Originally it had been decided to use the thread library within the Boost set of libraries. It provides an object-oriented wrapper around the C-style pthread allowing for a more flexible solution. To use the Boost thread library it is necessary to compile the libraries locally on the machine unlike some of other functionality which is available within the Boost libraries which are available in the pre-compiled headers. This is a very involved process as the compilation is not very intuitive. However critically it would seem that the dynamic link-library (or dll) generated is not compatible with the static library which has been written for the P3 Eye Camera. Therefore it was decided to abandon this library in favour of the multi-threading available within the standard Windows libraries.