Patent No. 5640493 Historical database training method for neural networks

 

Patent No. 5640493

Historical database training method for neural networks (Skeirik, Jun 17, 1997)

Abstract

An on-line training neural network for controlling a process for producing a product having at least one product property that trains by retrieving training sets from a stream of process data. The neural network detects the availability of new training data, and constructs a training set by retrieving the corresponding input data. The neural network is trained using the training set. Over time, many training sets are presented to the neural network. When multiple presentations are needed to effectively train, a buffer of training sets is filled and updated as new training data becomes available. The size of the buffer is selected in accordance with the training needs of the neural network. Once the buffer is full, a new training set bumps the oldest training set off the top of the buffer stack. The training sets in the buffer stack can be presented one or more times each time a new training set is constructed. An historical database of timestamped data can be used to construct training sets when training input data has a time delay from sample time to availability for the neural network. The timestamps of the training input data are used to select input data for use in the training set. Using the historical database, the neural network can be trained retrospectively by searching the historical database and constructing training sets based on past data.

Notes:

INCORPORATED BY REFERENCE

Incorporated by reference in their entirety herein are the following U.S. patents and patent applications (naming Richard D. Skeirik as the sole or one of the inventors):

U.S. Pat. No. 4,920,499, issued Apr. 24, 1990;

U.S. Pat. No. 4,884,217, issued Nov. 28, 1989;

U.S. Pat. No. 4,907,167, issued Mar. 6, 1990;

U.S. Pat. No. 4,910,691, issued Mar. 20, 1990;

Allowed U.S. patent application Ser. No. 07/103,014, filed Sep. 30, 1987;

Allowed U.S. patent application Ser. No. 07/103,047, filed Sep. 30, 1987; and

Pending U.S. patent application Ser. No. 07/333,536, filed Apr. 5, 1989.

SUMMARY OF THE INVENTION

The present invention is an on-line training neural network system and method for process control. The neural network trains by retrieving training sets from the stream of process data. The neural network detects the availability of new training data, and constructs a training set by retrieving the corresponding input data. The neural network is trained using the training set. Over time, many training sets are presented to the neural network.

The neural network can detect training input data in several ways. In one approach, the neural network monitors for changes in the data value of training input data. A change indicates that new data is available. In a second approach, the neural network computes changes in raw training input data from one cycle to the next. The changes are indicative of the action of human operators or other actions in the process. In a third mode, a historical database is used and the neural network monitors for changes in a timestamp of the training input data. Often laboratory data is used as training input data in this approach.

When new training input data is detected, the neural network constructs a training set by retrieving input data corresponding to the new training input data. Often, the current or most recent values of the input data are used. When a historical database provides both the training input data and the input data, the input data is retrieved from the historical database as a time selected using the timestamps of the training input data.

For some neural networks or training situations, multiple presentations of each training set are needed to effectively train the neural network. In this case, a buffer of training sets is filled and updated as new training data becomes available. The size of the buffer is selected in accordance with the training needs of the neural network. Once the buffer is full, a new training set bumps the oldest training set off the top of the buffer stack. The training sets in the buffer stack can be presented one or more times each time a new training set is constructed.

If an historical database is used, the neural network can be trained retrospectively. Training sets are constructed by searching the historical database over a time span of interest for training input data. When training input data is found, an input data time is selected using the training input data timestamps, and the training set is constructed by retrieving the input data at the input data time. Multiple presentations can also be used in the retrospective training approach.

The on-line training neural network can be used for process measurement, supervisory control, or regulatory control functions. Using data pointers, easy access to many process data systems is achieved. A modular approach with natural language configuration of the neural network can be used to implement the neural network. Expert system functions can be provided in the modular neural network to provide decision-making functions for use in control.

----------------------------------------------------

Although the foregoing refers to particular preferred embodiments, it will be understood that the present invention is not so limited. It will occur to those of ordinarily skill in the art that various modifications may be made to the disclosed embodiments, and that such modifications are intended to be within the scope of the present invention.