Introduction

With the launch of the Nintendo Wii game console back in 2006, few could have imagined the impact this console with its innovative use of motion control would have on the games market, least of all their competitors Sony and Microsoft. Four years later and Microsoft and Sony responded by launching their own offerings into the arena of motion controller interfaces.

Fig 1: Nintendo Wii Console with Wii Remote (Wiimote) and Nunchuk controls

Fig 2: Sony’s Playstation Eye Camera (left) & Move (right)

Sony’s offering is entitled Move, a motion-sensing game controller platform for the PlayStation 3 (PS3) video game console. The technology comprises of a handheld motion controller wand and a supplementary navigation controller. The PlayStation Move features an orb at the head which can glow in any of a full range of colours using RGB light-emitting diodes (LEDs) with the colour dynamically selected to distinguish it from the environment. The coloured light serves as an active marker, the position of which can be tracked along the image plane by the PlayStation Eye camera. The uniform spherical shape and known size of the light also allows the system to simply determine the controller's distance from the PlayStation Eye through the light's image size, thus enabling the controller's position to be tracked in three dimensions with high precision and accuracy. A pair of inertial sensors inside the controller, a three-axis linear accelerometer and a three-axis angular rate sensor, are used to track rotation as well as overall motion. An internal magnetometer is also used for calibrating the controller's orientation against the Earth's magnetic field to help correct against cumulative error (drift) by the inertial sensors. The inertial sensors can be used for dead reckoning in cases which the camera tracking is insufficient, such as when the controller is obscured behind the player's back.

Fig 3: Microsoft Kinect (formally Project Natal)

Microsoft's Kinect for its XBox 360 is more radical than the other two controllers by Sony and Nintendo as it dispenses entirely with the need for hand-held controllers or remotes altogether instead relying on a series of cameras and sensors to detect the position of players' movements, meaning that controlling a racing car can be achieved by simply holding your hands in a ten-to-two position and turning an imaginary steering wheel left and right. The device features an "RGB camera, depth sensor and multi-array microphone running proprietary software" which provides full-body 3D motion capture, facial recognition, and voice recognition capabilities. The depth sensor consists of an infrared projector combined with a monochrome CMOS sensor, and allows the Kinect sensor to see in 3D under any ambient light conditions. The sensing range of the depth sensor is adjustable, with the Kinect software capable of automatically calibrating the sensor based on gameplay and the player's physical environment, such as the presence of furniture. It is capable of identifying and tracking body parts to within a 4-centimetre cube in space, every 10 milliseconds, using just 10 to 15 per cent of the Xbox's computing resources. Kinect's body-recognition algorithms were trained to recognise body parts by running through millions of images, both human and computer-generated. While the training focused on poses that might be used in games, the system was also taught to recognise more mundane postures and gestures. This gives developers access to a palette of 20 skeletal joints, tracked in 3D, with which to create poses and gestures that the system will respond to.

Both Sony and Microsoft will hope to tap into the lucrative market developed by the Nintendo Wii which widened the demographic for games, attracting the emerging casual market which neither the PlayStation 3 nor Xbox 360 have yet managed to do with their more conventional and perhaps less accessible and less intuitive multiple-button gamepads.

Back in May 2010, it was with this in mind that I decided for my final project to build my own lightweight motion-capture system that could be used as a controller-less interface system similar to Microsoft's Kinect to control a character in a game by mapping the position of the player's head and hands in a 1:1 representation with a 3D game character, without the need for any special setup (chroma-key background) or prior marking of joints.