Patent No. 6254536 Method and apparatus for measuring and analyzing physiological signals for active or passive control of physical and virtual spaces and the contents therein

 

Patent No. 6254536

Method and apparatus for measuring and analyzing physiological signals for active or passive control of physical and virtual spaces and the contents therein (DeVito, Jul 3, 2001)

Abstract

A method and apparatus are disclosed for measuring and performing real-time FFT analysis of bioelectrical signals such as (EEG and EMG) for the control of systems. Passive and active interaction with various electronic media such as video games, movies, music, virtual reality, and computer animations is also disclosed. A method and apparatus for detecting the presence of a subject in a controlled area and for controlling real or virtual spaces is also disclosed.

Notes:

REFERENCE TO APPENDIX

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by any one of the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.

FIELD OF THE INVENTION

The present invention relates generally to the field of control systems based upon the real-time analysis of electrical measurements of physiological parameters, such as brainwaves (EEG) from an animal or electrical signals from a plant or other subject. In particular the present invention relates to the measurement and real-time analysis of bioelectrical signals for interaction with electronic media, such as motion pictures, digital video, video games, next-generation two and three dimensional games, virtual environments, and virtual world manipulations and other interactive environments. The principals of the present invention may however be extended to the control of other systems such as alarm systems and control of surveillance cameras.

BACKGROUND OF THE INVENTION

Sensed physiological parameters, and in particular bioelectrical signals (also called "bio-signals") such as brainwaves (EEG) and muscle signals (EMG) have been used to control electrical devices such as lights, music, game boards and children's toys. Previous control methods have relied upon threshold detection schemes wherein the voltage level of a band-limited signal exceeds a certain level (Hartzell et. al., U.S. Pat. No. 4,949,726), or on evoked potentials (Abdallah, U.S. Pat. No. 5,310,195), or on action potentials (Crawford Jr., U.S. Pat. No. 4,158,196). Other previous control methods alternatively calculate the peak power value using a Fast Fourier Transform (the "FFT") and determine whether the peak power exceeds a predefined threshold or the amount by which the threshold is exceeded for control. One drawback of those methods is that the person controlling the devices must learn how to generate particular signals in order to affect control, or must be provided with particular kinds of external stimuli. For example, in an alpha wave controlled system, the subject must learn to relax to reduce the amplitude of the brain (EEG) signals in the 8-12 Hz band. Another drawback of such threshold detection schemes is the susceptibility of the system to spurious signals such as muscle movement, for example, eye movement. Previous methods use specialized dedicated links between the subject and the controlled system.

As a result, there is a need for an improved method and apparatus for detecting and analyzing physiological signals, for utilizing the results of the analysis that does not require significant user training, and for utilizing the results of the analysis for the passive or active control of physical and virtual spaces and the contents therein.

SUMMARY OF THE INVENTION

Broadly, the present invention is directed to sensing physiological signals corresponding to the state of the subject that is being monitored, processing those signals in real time to develop a set of control parameters representative of the physiological state of the being, and using those developed control parameters, by means of a set of predefined relations (such as a look-up table or logic machine) that operates to map selected ones of the developed control parameters to command codes that are output to operate a control system. The mapping function may be user defined and/or variable in the case of certain control systems, or predetermined and fixed in the case of other control systems, and may include simple threshold or range comparisons, or more complex logical comparisons involving more than one control parameter.

A method in accordance with one aspect of the invention discards bio-signals having an amplitude outside a predetermined range, calculates the power spectrum in each of one or more predetermined bands, and calculates the total power across a plurality of the bands. The median frequency in each band and the relative power of each band are also calculated. Calculated values are compared with one or more previously calculated values from which a reliability factor for the calculated values are determined.

An apparatus in accordance with one aspect of the invention includes one or more physiological sensors, an analog-to-digital converter, a computer, and an interface to a controlled system. The analog-to-digital converter converts electrical signals produced by the sensors in response to physiological changes in the subject into digital values. The digital values are analyzed by the computer to provide relative power values in one or more of a plurality of frequency bands and median frequency values for one or more of a plurality of the frequency bands. The computer provides control signals to the controlled system based upon one or more of the relative power values or the median frequency.

As will be explained in more detail in the following discussions, the developed control parameters and predefined mapping relations may be selected by the user or designer of the control system, and such control systems may be command code responsive operating systems. Motion pictures, digital video, video games, next generation two and three dimensional games, virtual environments, and virtual world manipulations are some examples of interactive environments that may be controlled or influenced by the control system command codes. Additionally, alarm systems and surveillance cameras may be activated by the command codes.

A presence detecting function which senses the relationship of a subject to a predetermined space may be used to select an object or appliance to be controlled within a space. The control signals may be used to affect objects e.g., a compact disk player, or to affect the environment, e.g., lighting, within or without the controlled space. The system may be used to control a physical space, such as a home, or a virtual space, such as in a computer generated environment, or a combination of physical and virtual spaces.

An apparatus in accordance with one aspect of the invention includes at least one selector connected to one or more presence detectors for selecting a subject satisfying a predetermined relationship with the presence detectors. A control processor is connected to the selector and has an input for receiving bio-signals from the selected subject. The control processor includes an output for providing control signals to the objects under control.

A method in accordance with one aspect of the invention evaluates the relationship of a subject to a control space. Bio-signals sensed from a subject satisfying a predetermined relationship with the space are selected for control of at least one object or environmental parameter. The bio-signals are converted into one or more frequency domain parameters, one or more tests are performed on a preselected one or more of the frequency domain parameters according to preselected criteria, and control signals are provided according to at least one test result.

---------------------------------------

Although several embodiments of the present invention have been described using the IBVA transmitter and receiver, it should be appreciated that the present invention is not limited to such radio frequency carrier systems as it also contemplates directly wired electrodes, and other wireless systems such as an infra-red link.

From the foregoing description it will be apparent that improvements in physiological parameter based control systems and methods have been provided to improve the reliability and eliminate the need for subject training in active and passive systems. While preferred embodiments have been described, it will be appreciated that variations and modifications of the herein described systems and methods, within the scope of the invention will be apparent to those of skill in the art. Accordingly, the foregoing description should be taken as illustrative and not in a limiting sense.