Patent No. 6498618 Dual reality system

 

Patent No. 6498618

Dual reality system (Wilson, et al., Dec 24, 2002)

Abstract

An apparatus for merging real and virtual images in which points in three dimensions of an actual scene are obtained and selected in a random fashion and in a number sufficient to track the movement of the three dimensional scene relative to an observer wearing the apparatus. Updating of selected points occurs responsive to movement of a video camera pair observing the actual scene. The position and orientation of the unit housing the video camera pair is utilized to properly merge a generated virtual image with the real image. Selection of points and orientation of the unit relative to a three dimensional real image are obtained in real time. Microprocessor-based apparatus is utilized to perform the method of the invention.

Notes:

BACKGROUND

The present invention relates to personal display systems and more particularly to a display system which uniquely combines both virtual and real images in "real time".

Virtual reality systems are presently in widespread use and typically comprise a headset display which presents an image which is generated to simulate a real image (hence, "virtual" reality). Virtual reality systems lack the capability of presenting real images and especially lack the capability of presenting virtual images combined with real images and especially in "real time".

SUMMARY

The present invention provides a dual reality (DR) system which is characterized by providing apparatus and a process in which computer generated objects and images similar to those created by virtual reality systems, appear to the participant in the real world.

Similar to virtual reality, the dual reality environment is presented to the observer by way of a headset or visor which, similar to the virtual reality visor, provides a virtual image, and, in addition thereto, presents a stereoscopic view of the real environment through the employment of a pair of cameras coupled to the visor through a computer to present a stereoscopic view of the real environment to the participant through the visor.

The computer performs triangulation calculations which determine the distances from the visor to the selected navigational points to determine the position and orientation of the visor, which process will hereinafter be referred to as "visual navigation".

As one example, a computer generated object, such as a ball, viewed through the dual reality visor appears to float in front of the wearer. The observed surroundings are coupled to the visor display from the cameras and present a view of the actual surroundings as opposed to a computer generated display (i.e., a computer generated display of the actual surroundings.

The system employs environment mapping which can, for example, generate an image of the ball which appears to bounce on a real table in front of the participant or against real walls of a room that the participant is observing.

The dual reality system and method utilizes three basic components which include: (a) a visual navigation technique which determines the location and orientation of the dual reality system in a real three-dimensional space utilizing random points selected from an observed field of view; (b) the dual reality superimposition technique superimposes computer generated objects with images of the real three-dimensional space. An observer wearing a dual reality headset employing visual navigation can walk around a computer generated object in his or her real surroundings, observing the object from all sides. Hereafter, computer generated objects and images will be referred to as supplemental objects or collectively as supplemental reality; (c) environment mapping is the process by which supplemental objects are caused to "interact" with the real environment and with the observer identifying surfaces (such as walls, tables, panels, hands, etc.) in three-dimensional space wherein mathematical rules govern the interaction of supplemental objects with objects or surfaces in the real environment.

It is therefore one object of the present invention to provide a display which presents to an observer a dual reality image in which a computer generated image is superimposed with and reacts with a real image.

Another object of the present invention is to provide a display to an observer in which random points selected from a field of view are utilized to determine the location and orientation of the observer in a real three-dimensional space.

Still another object of the present invention is to provide a dual reality image presented to an observer by way of a visor worn by the observer, which dual reality image presents a computer generated image (virtual image) interacting with an actual three-dimensional image.

Still another object of the present invention is to provide a dual reality display presenting a virtual image interacting with a real, three-dimensional image wherein the interaction is obtained in accordance with pre-determined mathematical rules which govern the interaction.