Artificial Neural Networks (ANN) and Virtual Computer Laboratory

Ali Setoodehnia

Kean University, Union, NJ

Hong Li

The City University of New York, Brooklyn, NY

www.askli.com

Abstract

Virtual Computer Laboratory (VCL) is developed to address the capability of the Artificial Neural Networks (ANN) and the Data Simulations. The client-server construction of VCL was developed using Java Applet [1]. The VCL is accessible through ASKLI learning center. A sample demo is available at **www.askli.com/xor.htm**. The software simulation demo shows the method of construction of mathematical structure of dynamic system, which is observable only from its input and output with large or small number of input variables.

Introduction

The ANN is basically composed of many processing elements (neurons) with specific topology of weighted interconnections among these elements. And a learning law provides an algorithm for updating the connection weights. From the system modeling point of view, ANN can be thought as a highly nonlinear system, which have great potential to construct mathematical model using input/output measurement data. The ANN are now of great interest to system engineer, forecasting, pattern recognition, image processing, and many other areas of engineering and science due to their learning ability. The approximation capability of ANN can be divided into supervised and self-organized networks. In this paper the on-line data simulations for supervised back propagation and Self-Organized Modified Group Method Data Handling (MGMDH) models are addressed.

The possibility to use remote computers for the data analysis represents an important goal of the VCL. In a ‘virtual laboratory’, students and researchers can access the on-line scientific data analysis from anywhere. The VCL presents some of the ANN simulation results. The following sections present the VCL Client-Server and ANN models analysis and simulations.

Java Language is used to develop the Graphical User Interface (GUI) of ANN network. The Java Applet was created so that they can be easily accessible through the Internet with Java enabled browsers. The software includes the client application, providing visual interface for specific user-defined task. The user selects one out of the available options, supplies data for the simulation procedure. The software also includes the main server, providing data analysis result to the client. The calculated results can be visualized in the graphical user interface and stored into the user’s local disk.

ANN Simulations at Virtual Computer Lab:

The following procedure is used for VCL and running an experiment:

- Enter number of input variables in the Text Field, click Continue. It will display a Text Fields for number of nodes in each layer.
- Enter number of nodes for each layer, click Architecture to display the graphical architecture of ANN. At this point all of the connections are initialized with random numbers between [-1,1]. Click Data to display Text Area for training data.
- Type all of your data in Text Area, then click Parameters to display the number of Choices for Slope, Learning factor, Momentum, threshold, and number of iterations.
- Enter the slope, learning factor, Momentum, threshold, and select the number of iterations, and then click Run for the simulation.
- The total sum of square error along with the plot of error function is displayed for the user approval. The plot is two dimensions, the vertical axis is for error and the horizontal axis is for iterations. If the error becomes less than or equal to the threshold, the plot will stop, otherwise, user might need to change some of the parameters or number of nodes / layers and Run again.
- If result is satisfactory, click the Model to display the results. The result is the final connections weights for the above architecture.

Step-1 Get access to the Internet browser.

Step-2 Type VCL’s URL address to display general information index.html

page.

Setp-3 ANN simulations. Currently, the supervised and self-organized ANN

networks are available onwww.askli.com. For simplicity, if the

supervised network was selected, follow the following steps:

Similar VCL procedure is used for Self-Organized ANN network

Simulations:

Following are two of our simulation results, which are associated with the mathematical analysis given in this paper.

XOR-DEMO

1-**Type www.askli.com** **2- Press Current Research**

**3-Construct the model**** 4. Press run.**

**MGMDH DEMO:**

**1.Type www.askli.com/gmdh.htm 2. Follow the instructions**

Basic Principle OF ANN Network:

The structure we proposed for nonlinear dynamic system identification is a modified Feedforward Multilayer Perceptrons (FMP) [2,3], which has dynamic filtering capability.with respect to the threshold, it will correctly predict the output of an unknown Auto Regressive (AR), Moving Average (MA), or ARMA process when adequate samples of past input values are provided. This model is capable of predicting the output of chaotic systems. Moreover, this structure can be adapted for static systems by disconnecting some of the weight connections as well. In general, the modified structured is called ARMA Local FMP. For simplicity, the structure of one perceptron is shown in Figure-1.

Figure-1: Structure of one Perceptron

Where,

Xij*l(p) = åaijdl Xij*l(p-d) + åbijdl Xil-1(p-d) (1)

Xjl(p) = f(Zjl(p)) (2)

Zjl(p) = åXij*l(p) + ånjdl Xjl(p-d) (3)

Where l=1,…,L; p=1,…,T; j=1,…,Nl ; i=1,…,Nl-1 , L , T, Nl , are used for number of layers, number of patterns, and number of nodes in lth layer. X is the output of nonlinear sigmoid function for all layers and is used as input to ARMA. X* is the output of ARMA, 'a' and 'b' are the connection weights of the ARMA, and 'n' is the feedback at each node. A node is a linear combination of the input/output signals. The gradient descent method was used for updating the weights 'a', 'b', and 'n' for all layers.

Basic Principle of Modified GMDH

Group Method of Data Handling (GMDH) network is the Least Square Estimation (LSE) originally developed by Ivakhnenko at the Institute of Cybernetics of Ukrainian Academy of Science in 1966 [3,4]. This network models the input/output relationship of unknown system using a multilayer perceptron-type structure. This is a self-organized method based on sorting-out of gradually complicated models and their evaluation by external criterion on data samples.

For the system that has no prior information available, a method is needed to determine the mapping between the system input and output. The difficulty with system modeling methods [1,2,6] is the size space function, which creates a need for reliance of the algorithm on large amounts of prior knowledge or strong assumptions about the process that might not hold. It is necessary therefore, to develop an algorithm, which does not depend on this type of knowledge, but relies mostly on the observed data behavior and could incorporate this knowledge when it is available. However, the VCL-GMDH software implement the multilayer self-organized perceptron method for cases that the size of input vector and data set becomes very large; the ARMA-LFMP model might have very slow convergence rate.

Applications

The ANN can be used in many different fields. Some of the applications are as listed as [2,3,4,7]: Manufacturing, Ecological System, Medical, Environmental, Economical, Acoustic and Seismic analysis.

Summary

This paper has demonstrated the feasibility of VCL ANN Supervised and Self Organized network in modeling nonlinear static and dynamic systems for small or large data sets. The results of computer simulation with different nonlinear functions indicate that the ANN and MGMDH are capable of constructing mathematical model of unknown dynamic system. These networks can be used in many applications such as like pattern recognition, prediction, and chaotic systems. The Virtual Computer Laboratory was designed and implemented using Java Applet programming language. The VCL can be used as online data modeling services. Students and researchers can access to cutting-edge techniques of scientific data analysis.

Reference

- Deitel H.M, Deitel P.J., Java How to Program, third edition, Prentice Hall, 1999
- Astrom, A.J. and Wittenmark B., Adaptive Control, Addison-Wesley, (1989)
- Hecht-Nielson, R., Neurocomputing, Addison-Wesley, (1990)
- Ivakhnenko, A.G., “The Group Method of Data Handling-Arrival of the method of Stochastic Approximation”, Soviet Automation Control, V13, No. 3, (1966)
- Ivakhnenko, A.G., “Information Processing GMDH”, Soviet Automatic Control, (1990)
- Soderstorm, T., and Stoica, P., System Identification, Prentice Hall (1989)
- Stepashko, V.S., “ A GMDH Algorithm for Two Level modeling of multidimensional Cyclic Processes”, Soviet Automatic Control, (1988)

Updated 12/11/2005

WebMaster Bob Freid

WebMaster Bob Freid