We have developed a multi-player virtual-reality game platform, which is composed of acceleration sensors and transmitters/receivers of ultrasonic and radio frequency signals. The game players operate physical devices in the real world, and the positions, orientations, and velocities of the devices are computed and reflected in the virtual devices of the game world.
In a tabletop computer, the user interacts directly with a touch-sensitive screen instead of using a keyboard and a mouse. We are integrating the tabletop computing with realistic 3D visualization technology. We have developed a few game prototypes, which provide intuitive interfaces for game users.
Shadows produce very important effects in rendering a 3D scene. Light sources in the real world always have their sizes so they will create the soft shadows. In our research, computation of soft shadows is restated as a visibility problem for each point in the scene: how much of the light sources can be seen from the point? Our algorithm can generate gorgeous soft shadows in real-time.
The next-generation graphics hardware supports tessellation. As a result, per-vertex displacement mapping integrated with tessellation will be widely used in real-time applications. We are developing a novel 3D displacement mapping technique based on mean curvatures of surfaces. Our method can reconstruct self-occluded surfaces and high-quality silhouettes.
Generally, collision detection is the major computational bottleneck, and so simplified meshes or bounding volumes have been used to reduce the collision detecting time. Those approaches are fast but cannot efficiently handle the deformable objects and the dynamic scenes. The Media Lab has developed a new technique for image-space approach to collision detection using GPU, which does not construct the bounding volume hierarchy or simplify object meshes.
Physics is becoming the essential element in the game field. Game physics engines usually handles rigid body dynamics and constraint dynamics. The next generation game consoles will include deformable object physics to implement fluid, hair, cloth, etc. Along this trend, the Media Lab is focusing on designing and implementing physics engines that handle rigid body and deformable body together.
Simulations of fluids such as water, smoke, and fire are essential in computer graphics applications. The Media Lab has focused on the special kinds of fluids-objects interaction, where fluids can leak through non-waterproof objects such as cloth and sponge. We also have focused on the issues of the Rayleigh-Taylor instability for a time-dependent pressure boundary condition. For real-time applications, we are focusing on the Saint Venant equations instead of the Navier-Stokes equations.
For a home-service robot with a laser range scanner or a 3D camera, we are developing a real-time 3D map building algorithm. The point cloud data are organized in an octree, and planes such as walls and floors are extracted using the parallel processing power of GPU.
For young children's edutainment, a novel digital board game platform has been designed and developed, which consists of RFID tag-attached objects and an RFID reader board for recognizing the identities, locations and movements of the objects. The prototype games built on the platform prove the educational effectiveness brought by the natural interfaces. The RFID-based platform can be a useful tool for the next-generation board games.
A fast rendering algorithm for real-time animation of large crowds has been developed, which is essential for video games with a large number of non-player characters. The proposed approach leaves the minimal work of rendering to CPU, and makes GPU take all the major work, including LOD assignment and view frustum culling, which have been the typical tasks of CPU. By offloading the rendering overhead from CPU, the approach enables the CPU to perform intensive computations for crowd simulation.
Terrain rendering is the essential part for real-time rendering such as flight simulation. We have implemented a terrain rendering algorithm using ray-casting with pixel shader 3.0. In the experiments with nVidia 6800 Ultra, rendering of 1025x1025 height map terrain is performed in 70 fps without any mesh simplification.
An efficient technique has been developed to enhance the realism of character animation by adding muscle dynamics. Focusing on the isometric contraction of muscles, the proposed algorithm takes normal mesh and clenched mesh, and uses the disparity between them to simulate the skin vibration. The skin simulation algorithm is integrated with an example-based skinning, and shows real-time performance. The proposed approach proves to be useful for animating popping dance.
In order to depict realistic human motion and endow it with controllability, we have developed methods synthesizing motions with existing motion clips. Given an abundant set of motion clips, we organize the motion data into continuous motion spaces which are exploited in decoupled parameterized motion synthesizer. In the motion synthesizer, the user is able to create new motions by specifying the properties of both upper body and lower body of the character.
Robot is widely used in the production lines of factory, but is largely incapable of helping housework, where recognizing and analyzing the environment, and making a decision what to do should be performed in real-time. The Media Lab's research work in robotics is focused on real-time environment modeling and accessibility analysis. Especially, the accessibility analysis algorithm is implemented using GPU, and shows superior performances.
The Media Lab has worked in biomedical engineering field where multi-resolution analysis is adopted for designing elastic braces. When a joint is bent, the brace stuck onto it is accordingly deformed, and strain energy is stored in the brace material. The strain energy is calculated using strain energy density functions. For effective calculation, mesh simplification and surface parametrization techniques are innovatively applied.