Patent No. 6466185 Multi-planar volumetric display system and method of operation using psychological vision cues

 

Patent No. 6466185

Multi-planar volumetric display system and method of operation using psychological vision cues (Sullivan, et al., Oct 15, 2002)

Abstract

A multi-planar volumetric display system and method generate volumetric three-dimensional images using a multi-surface optical device including a plurality of individual optical elements corresponding to a plurality of image depths. An image projector selectively projects images on respective optical elements to generate a volumetric three-dimensional image viewable in the multi-surface optical device. Psychological vision cues are added during generation of the three-dimensional image to enhance depth perception when viewing the three-dimensional image. A floating-image generator may project the three-dimensional image having enhanced depth to generate a second volumetric three-dimensional image viewable as floating in space.

Notes:

Multi-planar volumetric display system and method of operation using psychological vision cues. Filed August 2001, granted October 2002. Funded by DARPA. Modify what the patent says to use the data stream from the remote neural monitoring as its input data stream. This patent says it does what it does with a 195 MHz processor. Hmm, I think DARPA has some faster ones. Can be used to generate virtual reality or holographic displays from separated and processed EEG stream/s (separating various potentials and 'noise' and interpreting them). Basically the psychological cues that the patent makes reference to in order to generate the displays, would approximate what the Brain programmable display system does with evoked potentials to create a display.

BACKGROUND OF THE INVENTION

The present invention relates to three-dimensional (3D) imaging, and, more particularly, to a multi-planar display system using 3D anti-aliasing for generating volumetric three-dimensional images in space.

It is known that three-dimensional (3D) images may be generated and viewed to appear in space. Typically, specialized eyewear such as goggle and/or helmets are used, but such eyewear can be encumbering. In addition, by its nature as an accessory to the eyes, such eyewear reduces the perception of viewing an actual 3D image. Also, the use of such eyewear can cause eye fatigue which is remedied by limiting the time to view the image, and such eyewear is often bulky and uncomfortable to wear.

Thus, there is a need to generate volumetric 3D images and displays without the disadvantages of using such eyewear.

Other volumetric systems generate such volumetric 3D images using, for example, self-luminescent volume elements, that is, voxels. Before providing examples of such systems, it is important to distinguish the much abused term "voxel" from a 3D data element (referred to herein as a "tridel"). A voxel is the actual glowing point of light in a 3D display and is analogous to a pixel in a 2D display. However, a tridel is an abstract 3D data type. More specifically, voxels have positions that are integers (i, j, k) and only have the properties of color and brightness, whereas tridels are characterized by a set of parameters defined at a floating point location (x, y, z) in a virtual image space. Thus, in its most general sense, a tridel is a 3D data type any may encompass any number of application-specific data types. For example, if the tridel is used to define polygonal vertices of a 3D object then the data parameters of this abstract 3D data type are color (R, G, B) and visual opacity (A). As another example, if the tridel represents a data element of an image produced by a medical computed x-ray tomography ("CT") scanner, then the data parameter is x-ray opacity. In yet another example, if the tridel describes a thermonuclear plasma, then the data parameters might be plasma density, temperature, and average velocity of the plasma constituents.

From the foregoing, it will be understood that to produce an image, either 2D or 3D, each tridel must be mathematically processed into a pixel or voxel. This processing may include geometric transformations including rotation, scaling, stretching or compression, perspective, projection and viewpoint transformations, all of which operate on the x, y, z coordinates of the tridel. Further, in the process of determining the color and brightness of a pixel or voxel, tridels may be averaged together when there are many within the space of one voxel or interpolated between when there many pixels within the space of two tridels. The distinction between tridels and voxels will be more clearly appreciated upon consideration of the depth transformation discussed below for mapping the depth coordinate of a tridel into the voxel depth coordinate within the MOE device 32.

Turning to examples of other volumetric display systems known in the art, one example of a volumetric image system is the system of 3D TECHNOLOGY LABORATORIES of Mountain View, Calif., in which the intersection of infrared laser beams in a solid glass or plastic volume doped with rare earth impurity ions generates such voxel-based images. However, the non-linear effect that creates visible light from two invisible infrared laser beams has a very low efficiency of about 1%, which results in the need for powerful lasers to create a bright image in a large display. Such powerful lasers are a potential eye hazard requiring a significant protective enclosure around the display. Additionally, scanned lasers typically have poor resolution resulting in low voxel count, and the solid nature of the volumetric mechanism results in large massive systems that are very heavy.

Another volumetric display system from Actuality Systems, Inc. of Cambridge, Massachusetts, uses a linear array of laser diodes that are reflected off of a rapidly spinning multifaceted mirror onto a rapidly spinning projection screen. However, such rapidly spinning components, which may be relatively large in size, must be carefully balanced to avoid vibration and possibly catastrophic failure. Additionally, the size, shape, and orientation of voxels within the display depends on their location, resulting in a position-dependent display resolution.

Another volumetric display system is provided by NEOS TECHNOLOGIES, INC., of Melbourne, Fla., which scans a laser beam acousto-optically onto a rapidly spinning helical projection screen. Such a large spinning component requires a carefully maintained balance independent of display motion. The laser scanner system has poor resolution and low speed, drastically limiting the number of voxels. Additionally, the size, shape, and orientation of voxels within the display depends on their location, resulting in a position-dependent resolution. Finally, the dramatically non-rectilinear nature of the display greatly increases the processing requirements to calculate the different two-dimensional images.

Other types of 3D imaging system are known, such as stereoscopic displays, which provide each eye with a slightly different perspective view of a scene. The brain then fuses the separate images into a single 3D image. Some systems provide only a single viewpoint and require special eyewear, or may perform headtracking to eliminate eyewear but then the 3D image can be seen by only a single viewer. Alternatively, the display may provide a multitude of viewing zones at different angles with the image in each zone appropriate to that point of view, such as multi-view autostereoscopic displays. The eyes of the user must be within separate but adjacent viewing zones to see a 3D image, and the viewing zone must be very narrow to prevent a disconcerting jumpiness as the viewer moves relative to the display. Some systems have only horizontal parallax/lookaround. In addition, depth focusing-convergence disparity can rapidly lead to eyestrain that strongly limits viewing time. Additionally, stereoscopic displays have a limited field of view and cannot be used realistically with direct interaction technologies such as virtual reality and/or a force feedback interface.

Headmounted displays (HMD) are typically employed in virtual reality applications, in which a pair of video displays present appropriate perspective views to each eye. A single HMD can only be used by one person at a time, and provide each eye with a limited field of view. Headtracking must be used to provide parallax.

Other display systems include holographic displays, in which the image is created through the interaction of coherent laser light with a pattern of very fine lines known as a holographic grating. The grating alters the direction and intensity of the incident light so that it appears to come from the location of the objects being displayed. However, a typical optical hologram contains an enormous amount of information, so updating a holographic display at high rates is computationally intensive. For a holographic display having a relatively large size and sufficient field of view, the pixel count is generally greater than 250 million.

Accordingly, a need exists for high quality volumetric 3D imaging with computationally acceptable demands on processing systems and which has improved viewability and implementation.

In addition, in three-dimensional imaging, the use of discrete voxels renders portions of images to appear jagged due to pixelization, for example, for features at transitions between discrete depths in a volumetric 3D image. A need exists for a method which softens the transition between portions of a volumetric 3D image.

SUMMARY OF THE INVENTION

A multi-planar volumetric display (MVD) system and method of operation are disclosed which generate volumetric three-dimensional images. The MVD system includes a multi-surface optical device including a plurality of individual optical elements arranged in an array; an image projector for selectively projecting a set of images on respective optical elements of the multi-surface optical device; and a floating-image generator for projecting the first volumetric three-dimensional image from the multi-surface optical devices to generate a second volumetric three-dimensional image viewable as floating in space at a location separate from the multi-surface optical device.

Each of the plurality of the individual optical elements of the multi-surface optical device includes a liquid crystal element having a controllable variable translucency. An optical element controller is also provided for controlling the translucency of the liquid crystal elements, such that a single liquid crystal element is controlled to have an opaque light-scattering state to receive and display the respective one of the set of images from the image projector, and the remaining liquid crystal elements are controlled to be substantially transparent to allow the viewing of the displayed image on the opaque liquid crystal element.

The optical element controller rasters through the liquid crystal elements at a high rate during a plurality of imaging cycles to select one liquid crystal element therefrom to be in the opaque light-scattering state during a particular imaging cycle, and to cause the opaque light-scattering state to move through the liquid crystal elements for successively receiving the set of images and for generating the volumetric three-dimensional images with three-dimensional depth.

The image projector projects the set of images into the multi-surface optical device to generate the entire first volumetric three-dimensional image in the multi-surface optical device at a rate greater than 35 Hz to prevent human-perceivable image flicker. For example, the volume rate may be about 40 Hz. In one embodiment, for example, if about 50 optical elements are used with a volume rate of about 40 Hz, the image projector projects each of the set of images onto a respective optical element at a rate of 2 kHz.

The image projector includes a projection lens for outputting the set of images. The projector also includes an adaptive optical focusing system for focusing each of the set of images on the respective optical elements to control the resolution and depth of the projection of the set of images from the projection lens. Alternatively or in addition, the image projector includes a plurality of laser light sources for projecting red, green, and blue laser light, respectively, to generate and project the set of images in a plurality of colors.

In addition, a 3D anti-aliasing method is employed to smooth the portions of the projected images at transitions between optical elements in the multi-surface optical device. The anti-aliasing adjusts the display of voxels in a transition between optical elements, such that color values of the voxels are modified as a function of the distance of the voxels from the optical elements, to generate a smooth transition between the portions of the volumetric three-dimensional image.

-------------------------------

The MVD system 10 may be implemented using the apparatus and methods described in co-pending U.S. Provisional Patent Appln. No. 60/082,442, filed Apr. 20, 1998, as well as using the apparatus and methods described in U.S. Pat. No. 5,990,990 filed Nov. 4, 1996, which is a continuation-in-part of U.S. Pat. No. 5,572,375; which is a division of U.S. Pat. No. 5,090,789. The MVD system 10 may also he implemented using the apparatus and methods described in co-pending U.S. patent appln. Ser. No. 09/004,722, filed Jan. 8, 1998. Each of the above provisional and non-provisional patent applications and issued patents, respectively, are incorporated herein by reference. Accordingly, the invention has been described by way of illustration rather than limitation.