Identification of Systems Using Radial Basis Networks Feedbacked with FIR Filters

By the rising complexity and miniaturisation of the device's dimensions, the density of the conductors increases considerably. Referring to this, locally transient interactions between single physical values become apparent. Therefore, for the investigation and optimisation of integrated circuits it is essential to develop suitable models and simulation surroundings which allow for memory and time-efficient calculation of the behaviour. By means of the dynamic reconstruction theory and the radial basis functions nets the so-called black box models are provided. The description of black box models is derived from the input and output behaviour or so-called time series of a dynamic system. Concerning the time series, the black box model adapts its parameters via the extended Kalman filter. This paper provides a modelling approach that enables fast and efficient simulations.


Introduction
The increasing simulation time is a big limiting factor during the design and the development of HDI/HDP systems.Hence, it is more and more crucial to provide models which allow for quick and accurate simulations (Wiegand et al., 2007b,a;Stievano, 2002).The classical modelling approach, based on the transistor level circuits and the symbolic analysis, requires unreasonably long computation times.The black box modelling (BBM) via radial basis functions (RBF) nets for integrated circuits is an adequate solution that provides robust and fast simulations.In the following, a method is presented for developing mathematical models to reduce the very growing simulation times for transient analysis.This modelling approach is based on the dynamic reconstruction of dynamic systems.That is, a mathematical formulation needs to be found, that describes the dynamics of the system to be examined as accurately as possible without necessarily considering the physical characteristics of the circuit.
Correspondence to: C. Wiegand (christopher.wiegand@pb.izm.fraunhofer.de)In Sect. 2 the modelling flow for developing black box models is discussed, although the radial basis functions nets are the main focus of all considerations.Section 3 describes the architecture of the RBF nets.The parameter adaptation, i.e. the extended Kalman filter, is explained in Sect. 4. In Sect. 5 a NOR element with two inputs and three outputs is examined.Then, an example of a diode is considered and the RBF model is parameterised in such a way so as to allow for the temperature to be regarded as a further input.

Modelling flow
It is necessary for the developing of black box models by means of radial basis functions nets to choose a suitable time series (measurements or simulations), so that the model is as close to the original system as possible.Thus, the training set must comprise a high information content with which a reliable model can be provided.The next step in the modelling chain (see Fig. 1) is the embedding (Sauer et al., 1991) of this time series in the RBF net, i.e. the choosing of centres.This can be done with the help of the so-called locally regularised orthogonal least square (Chen et al., 1996,?;Chena et al., 1990a) or the K-Means-Cluster-Algorithm (Sing et al., 2003).Within this work the centres are determined by the clustering procedure, because the initialisation of the remaining parameters of the net can be determined via least square methods.Afterwards the parameter adaptation (parameter estimation) is achieved by means of the extended Kalman filter (EKF) (Haykin, 2001;Ahmida and Charef, 2002).The final step is the Validation, i.e., assuming that the error between the model and the system is in the expected order of magnitude, a robust simulation model is provided.If this case does not occur, it has to be assessed, whether the training signal was chosen appropriately since the centres were possibly not picked wisely, or the parameter adaptation diverged or remained in a local minimum.

Radial basis functions nets
With using RBF networks, it is possible to represent any continuous nonlinear function.Because of this important feature, they are utilised to model nonlinear dynamic systems and integrated circuits.Radial basis functions nets are special feedforward neural networks consisting of three layers with a hidden layer.
Input layer: definition of the number of inputs, i.e. within the regressor vector the number of inputs is fixed.
Hidden layer: signal processing via p radial basis functions (Gaussian) or so-called neurons.Furthermore the embedding of the data occurs just in the hidden layer.
Output layer: definition of the number of outputs.Regarding weighted transitions the neurons outgoing signals are added up within every output.
Figure 2 shows a schematic representation of a typical RBF network with multiple inputs and multiple outputs (MIMO).The response of a MIMO radial basis functions net can be formulated as follows: where φ i (ζ i ) denotes the i th gaussian activation function in the hidden layer and can be written as and x is the regressor vector and is denoted by Within the regressor vector the n inputs are defined; m and n describe the number of outputs and the number of basis functions; c i is the i th centre and represents the shape of the i th basis function.Equation 1 can be described in the following way: In a standard design, the RBF net provides a static input and output behaviour.For temporal processing, time delays have to be introduced to convert the static RBF network into a dynamic network.There are only very few possibilities to include memory into the architecture.For example discrete time FIR 1 /IIR-Filters 2 can be used instead of the weights or global feedback paths (Howlett, 2001a,b) could be introduced.A simple method to create dynamics can be expressed by: where d 1 and d 2 denote the input embedding dimension and the output embedding dimension.By making such changes, a state space model can be described (Wiegand et al., 2007a).
The first order derivation can be employed to transform the discrete time representation into a continuous state space model.

Parameter estimation
Generally, the parameters of a RBF net can be determined by a huge number by algorithms, for example, by the locally regularised least square algorithm with centre selection (Chena et al., 1990a;Chen et al., 1990bChen et al., , 1996)).Nevertheless, non-linear optimisation methods must then be employed to adapt the shapes i of the basis functions.Within this work, the extended Kalman filter (EKF) is used for parameter identification of non-linear systems.In accordance to this, all parameters are determined iteratively by the EKF, including those which are in the argument of the gaussian function.For the initialisation and the improvement of the convergence, the K-Means cluster algorithms is used to determine the centres.In addition, this cluster algorithm delivers an facile possibility for the calculation of the shapes (widths) of the basis functions, to be able to determine the weights by means of least square methods.

K-Means cluster algorithm
The algorithm is an iterative procedure to divide a set of training data into groups (cluster).The only available information is the training data and the number of the clusters.This is also known as unsupervised learning.A random choice of the cluster centres c(0) within the training set T serves as the starting point.Next, in every iteration step k, every set of trainig data is allocated to the nearest situated cluster centre and thus the subsets M i are obtained.The identification of the new centres is accomplished by means of average determination via the sets 1 FIR: Finite Impulse Response 2 IIR: Infinite Impulse Response Fig. 3. NOR-Circuit with two inputs (V A und V B ) and three outputs (V out1 , V out2 und V out3 ).The buffers of the respective outputs are four cascaded inverters.Also, the outputs are concluded with different complex impedances (Z i , where i∈[1, 2, 3]).
whereas N i denotes the number of elements which are in the set M i .An initialisation of the matrices can take place after the determination of the centres, i.e. after the last iteration step: With the calculated initial centres and the widths of the basis functions, it is possible to determine the weights, i.e., by utilising the least square methods.

Extended Kalman Filter
Iteratively, the EKF estimates the parameter vector where K k is the Kalman gain.
e=y− ŷ is the error between the observed outputs y (target vector) and the estimated outputs ŷ of the RBF network; H k = ∇ w f (x, w) is the matrix of the partial derivative of f (x, w).For this reason, when considering the j th output, the derivations of f j (x, w) are related respectively to the parameters c i , θ j i and i : ∂f j (x, w) where i is defined as R k and P k denote the variance of the measurement noise and the error covariance matrix.The error covariance matrix is a positive definite symmetric ν×ν matrix, where ν is the number of parameters being updated and can be determined by: According to Ahmida and Charef (2002), q 0 regulates the allowed random step towards the gradient.For fast parameter estimation, the training data is subdivided into smaller sets.These sets are used to adjust the parameters of the RBF net.Afterwards, a test is performed as well as an update of the regressor vector.

Simulations
In the following section, two different nonlinear systems are considered.
The first example corresponds to a two input and three output NOR-Circuit (see Fig. 3).In the second example a diode is envisaged, in which the ability of parameterisation of the RBF nets is illustrated.

Example 1: NOR Circuit
At first a NOR circuit is considered, which was built-up and simulated by using the Agilent Advanced Design System (ADS) environment.This circuit was implemented with two inputs and three outputs with different complex loads, from which a MIMO system arises.The circuit is implemented within 130 nm technology derived from an ADSimplemented Philips library.The radial basis functions net has been trained using a set of 1000 data samples of the trapezoidal input signals and the corresponding output signals.
The RBF net is developed with 15 centres (selected via the presented cluster algorithm) and the regressor vector is defined as follows:

Fig. 2 .
Fig. 2. Schematic representation of a multiple input and multiple output radial basis functions net.

Fig. 4 .
Fig. 4. Input and output signals of the simulated system and model.The output voltages marked with a hat denote the outputs of the RBF model.

Fig. 5 .
Fig. 5. Identity lines of the diode and the model for different temperatures.