9512.net

甜梦文库

甜梦文库

当前位置：首页 >> >> # Using neural network function approximation for optimal design of

Computers & Operations Research 30 (2003) 339 – 352

www.elsevier.com/locate/dsw

Using neural network function approximation for optimal design of continuous-state parallel–series systems

Peter X. Liua , Ming J. Zuob;? , Max Q.-H. Menga

a b

Department of Electrical and Computer Engineering University of Alberta, Edmonton, Alta., Canada T6G 2G7 Department of Mechanical Engineering, University of Alberta, 4-9 Mechanical Engineering Building, Edmonton, Alta., Canada T6G 2G8 Received 1 November 2000; received in revised form 1 July 2001

Abstract This paper presents a novel continuous-state system model for optimal design of parallel–series systems when both cost and reliability are considered. The advantage of a continuous-state system model is that it represents realities more accurately than discrete-state system models. However, using conventional optimization algorithms to solve the optimal design problem for continuous-state systems becomes very complex. Under general cases, it is impossible to obtain an explicit expression of the objective function to be optimized. In this paper, we propose a neural network (NN) approach to approximate the objective function. Once the approximate optimization model is obtained with the NN approach, the subsequent optimization methods and procedures are the same and straightforward. A 2-stage example is given to compare the analytical approach with the proposed NN approach. A complicated 4-stage example is given to illustrate that it is easy to use the NN approach while it is too di cult to solve the problem analytically. Scope and purpose The classical reliability theory assumes that the system and each component may only be in one of two possible states: working or failed. Thus, it is also referred to as binary reliability theory. A well-known reliability design problem under the binary reliability theory involves the determination of the number of redundancies in a parallel–series system which consists of N subsystems connected in series whereas each subsystem consists of a few components connected in parallel. In this paper, we consider the optimal design problem of a multi-state parallel–series system wherein both the system and its components may assume more than two levels of performance. Speci?cally, we assume that the state of each component and the system may be represented by a continuous random variable that may take values in the closed interval [0; 1]. An optimization model is formulated for the determination of the number of redundancies in order to maximize

?

Corresponding author. Tel.: +1-780-492-4466; fax: +1-780-492-2200. E-mail address: ming.zuo@ualberta.ca (M.J. Zuo).

0305-0548/02/$ - see front matter ? 2002 Elsevier Science Ltd. All rights reserved. PII: S 0 3 0 5 - 0 5 4 8 ( 0 1 ) 0 0 1 0 0 - 9

340

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

the system’s expected utility function. Because of the complexity of the optimization problem, we propose a neural network (NN) approach to approximate the objective function. The resulting optimization model is much easier to solve. Examples are given to illustrate the proposed approach. ? 2002 Elsevier Science Ltd. All rights reserved.

Keywords: Neural networks; Continuous-state system; Parallel–series system; Optimal system design; Multi-state system

1. Introduction A system consists of n components, each of which may perform a di erent function. One of the most important measures of the performance of a system is its reliability. The reliability of a system is de?ned to be the probability that the system will perform its functions satisfactorily for a certain period of time under speci?ed conditions. The traditional reliability theory assumes that a system and its components may only experience one of two possible states: working or failed. As a result, we call it binary reliability theory. Under the binary assumption, reliability as de?ned above is an excellent measure of the performance of systems. There exist several methods for designing systems with high reliability. These methods include using large safety factors, reducing the complexity of the system, increasing the reliability of constituent components, and using structural redundancy. Kuo et al. [1] provides an extensive coverage on optimal system design. A parallel–series system consists of N subsystems connected in series such that the system works if and only if all the subsystems work wherein subsystem i (1 6 i 6 N ) consists of Mi components connected in parallel such that the subsystem fails if and only if all the components in this subsystem fail. Fig. 1 shows such a parallel–series con?guration. The reliability of such a parallel–series system is expressed as: ? ?

N

Rs =

i=1

?1 ?

Mi

(1 ? pij )? ;

(1)

j =1

where pij is the reliability of component j in subsystem i. For such a system, a typical optimization problem involves ?nding the optimal number of parallel components in each subsystem either to maximize system reliability or minimize total system cost. The constraints for such problems are either resource limitations or reliability requirements. Resource limitations usually represent constraints of cost, weight, volume or some combinations of these factors. The reliability constraint imposes a minimum requirement of system reliability. In either case, the system design problem is a nonlinear integer-programming problem. Many algorithms have been proposed but none has proven to be superior over the others. Kuo et al. [1] surveyed and classi?ed optimization techniques related to nonlinear programming problems. They compared the pros and cons of the following optimization techniques: integer programming, transforming nonlinear to linear functions, dynamic programming, the sequential unconstrained minimization technique (SUMT), the generalized reduced gradient method (GRG), the modi?ed sequential simplex pattern search, and the generalized Lagrangian

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

Subsystem 1 1 In 2 Subsystem i 1 2 Subsystem N 1 2 Out

341

M1

Mi

MN

Fig. 1. The structure of a parallel–series system.

function method. Other examples of integer programming solutions to the redundancy allocation problem are presented by Misra and Sharma [2], and Gen et al. [3,4]. In recent years, genetic algorithms have been used for solving reliability based design problems, for example, see [5,6]. The binary assumption has served as a unifying foundation for the mathematical theory of reliability. However, in many real-life situations, a multi-state system model is needed to allow both the system and its components to assume more than two levels of performance. In a discrete multi-state system, it is assumed that the system and its components may each experience M + 1 possible states (M ? 1): 0; 1; : : : ; M , where 0 represents the completely failed state, M represents the completely working state, and others are intermediate states. For studies of discrete multi-state systems, users are referred to Barlow and Wu [7], Gri th [8], Xue and Yang [9], and Huang et al. [10]. Zuo et al. [11] considered the redundancy allocation problem for discrete multi-state parallel–series systems and provided a heuristic algorithm for solving the optimal design problem. In a continuous multi-state system, it is assumed that the states of the system and its components may each be represented by a continuous random variable de?ned in the closed interval [0; 1], where 0 represents the complete failure state and 1 the completely working state. For studies of continuous multi-state systems, readers are referred to Ross [12], Block and Savits [13], and Cappelle and Kerre [14]. In a multi-state system, be it continuous or discrete, the de?nition of reliability as given under the binary assumption is no longer valid. Di erent measures of system performance need to be de?ned. In this paper, we consider the redundancy allocation problem for continuous-state parallel–series systems. The system utility function is used to measure the performance of the system. The NN approach is used to solve the nonlinear integer optimization problem. Examples are given to illustrate the advantages of the NN approach.

2. The continuous-state system design model Consider a parallel–series system as shown in Fig. 1. There are N subsystems connected in series. Subsystem i (for 1 6 i 6 N ) has Mi components connected in parallel. The state of a component can be represented by a continuous random variable de?ned in the range from 0 to 1, inclusive. Before introducing the optimal design model of the continuous-state parallel–series system, we provide the

342

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

following list of notation: range of possible state, = [0; 1], where 1 indicates perfectly functioning state and 0 the completely failed state s state index, a deterministic value, s ∈ Xij state of component j in subsystem i, a random variable, 0 6 xij 6 1 x the vector representing the states of all components in the system, x = (xij ); i = 1; 2; : : : ; N ; j = 1; 2; : : : ; Mi N total number of subsystems of the system Mi total number of components in the ith subsystem ’(x) system structure function, or system state as a function of component states, a random variable, 0 6 ’(x) 6 1 fij (s) probability density function of xij fi (s; Mi ) probability density function of the state of the ith subsystem f(s; M1 ; : : : ; MN ) probability density function of (x), the state of the system (s) utility function of the system when it is in state s Cij cost of the j th component in the ith subsystem CT total budget available for the system to be designed In addition, we also use the following assumptions: 1. The states of the components in the same subsystem are independently and identically distributed (iid). 2. The utility function of the system, (s), is known. 3. The state probability density function fij (s) of each component is known. According to Barlow and Wu [7], the state of a parallel system is equal to the state of the best component in the system while the state of a series system is equal to the state of the worst component in the system. As a result, the state of the parallel–series system shown in Fig. 1 can be expressed as (x) = min

16i6N 16j 6Mi

max xij ;

(2)

where xij has probability density function fij (s) for 0 6 s 6 1. Using Eq. (2), we can evaluate the following probability: ? ?

N

Pr(’(x) ? s) =

i=1

1

N

s

fi (t; Mi ) d t

=

i=1

?1 ?

Mi

s

j =1

0

fij (t ) d t ? for 0 6 s 6 1:

(3)

It is obvious that Pr(’(x) ? s) as shown in Eq. (3) is a function of s; M1 ; M2 ; : : :, and MN . 1 Since Pr(’(x) ? s) ≡ s f(t; M1 ; M2 ; : : : ; MN ) d t , we obtain the following expression of the probability density function of ’(x): ? ? ?? Mi N s d d ?1 ? f(s; M1 ; M2 ; : : : ; MN ) = ? Pr(’(x) ? s) = ? ? fij (t ) d t ?? : (4) ds d s i=1 0 j =1

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

343

The expected utility of the system is: U (M1 ; M2 ; : : : ; MN ) =

1 0

(s)f(s; M1 ; M2 ; : : : ; MN ) d s

1 0

? (s) ?

?

=?

d ? ?1 ? d s i=1 j =1

N

?

Mi

s 0

??? fij (t ) d t ??? d s: (6)

The expected system utility, U (M1 ; M2 ; : : : ; MN ), depends on the following factors: (1) the number of sub-systems, N ; (2) the number of components in each subsystem, M1 ; M2 ; : : : ; MN ; (3) state probability density function of each component, fij (s); and (4) utility function of the system when it is in the state s; (s). The number of subsystems, N , is usually determined by the system function required. In this paper, we also assume that both fij (s) and (s) are known. Thus the optimal design problem is concerned with ?nding the optimal values of M1 ; M2 ; : : :, and MN to maximize U (M1 ; M2 ; : : : ; MN ) subject to the cost constraints, as shown below: ? ? ? ??? Mi N 1 s d ?1 ? Maximize : U (M1 ; M2 ; : : : ; MN ) = ? (s) ? ? fij (t ) d t ??? d s d s i=1 0 0 j =1

N Mi

Subject to :

i=1 j =1

Cij 6 CT :

This optimization model includes both integration and di erentiation in the objective function. The objective function is very complicated and it is very di cult to use a classical optimization algorithm. This situation arises when (1) the number of subsystems, N , is large; (2) fij ’s in the same subsystem are not identical; and= or (3) fij is not a simple distribution. In addition, in some cases, the component state distribution function may have to be expressed in an empirical form and, as a result, no analytical expression of U (M1 ; M2 ; : : : ; MN ) is available. In the following section, we propose to use the NN approximation to solve this optimization problem. 3. The NN approximation We know that Pr(’(x) ? s) is a nonlinear function of variables, s; M1 ; M2 ; : : : ; MN . For convenience, we de?ne that, ? ?

N

g(s; M1 ; M2 ; : : : ; MN ) ≡ Pr(’(x) ? s) =

i=1

?1 ?

Mi

s

j =1

0

fij (t ) d t ? :

(7)

Thus, the objective function becomes, d (g(s; M1 ; M2 ; : : : ; MN )) d s: (8) ds 0 If g(s; M1 ; M2 ; : : : ; MN ) is only a linear combination of standard sigmoidal functions, U (M1 ; M2 ; : : : ; MN ) has a simple expression as to be demonstrated through examples later. This observation stimulates us to use neural networks to approximate g(s; M1 ; M2 ; : : : ; MN ). Because of the monotonic U (M1 ; M2 ; : : : ; MN ) = ? (s)

1

344

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

s M1 2 MN N+1 c θc 1 W k θk αc

1

α1 θ1

Σ

? g

Fig. 2. The structure of a feedforward neural network with one hidden layer.

and nonlinear characteristics of neurons, sigmoidal neurons for example, NNs are universal approximators. Cybenko [15] and Aourid and Do [16] show that a ?nite linear combination of sigmoidal functions can approximate any continuous function of n real variables with support in the unit hypercube to any degree of accuracy. As a result, it is possible to use a feedforward NN with a single hidden layer as shown in Fig. 2, to approximate g(s; M1 ; M2 ; : : : ; MN ). Generally, analytical expressions of multiplayer NNs (the activation functions of two or more layers are nonlinear) are generally impossible to obtain. However, for the NN model in Fig. 2, only the single hidden layer is of nonlinear functions and the activation function of the output layer is a pure linear sum operator. Thus, the output of the NN is actually a weighted sum of a ?nite number of sigmoidal functions. Once the training is completed, we know the number of hidden neurons and all the weights and bias factors associated with each neuron. An analytical expression of the NN output, i.e., the approximate system distribution function in this paper, can be written as

c

g ?(s; M1 ; : : : ; MN ) =

k =1

k

(WkT y + ?k )

(9)

where y = [s; M1 ; : : : ; MN ]T is the input vector; (u) = 1= (1 + e?u ) is the activation function of the hidden units; Wk = [wk 1 ; wk 2 ; : : : ; wk (N +1) ]T is the weight vector associated with the hidden neuron k ; c is the number of hidden units, k is the weight between the hidden unit k and the output neuron, wki is the weight between the ith input neuron and the k th hidden neuron, and ?k is the bias factor of the k th hidden neuron. From Eq. (9), we can also see that no matter how complicated g(s; M1 ; M2 ; : : : ; MN ) might be, the approximate analytical expression of system distribution function, g ?(s; M1 ; M2 ; : : : ; MN ), is always a linear combination of a ?nite number of sigmoidal functions. As stated earlier, the approximate ? (M1 ; M2 ; : : : ; MN ) constructed from g objective function U ?(s; M1 ; M2 ; : : : ; MN ) is usually solvable. Thus, the subsequent optimization procedure is exactly the same for various component state distribution functions. For simple and analytically solvable problems, NN approximation is unnecessary. However, for large, complicated and analytically unsolvable problems, the NN approximation can be as accurate as desired. The basic idea to use NNs in this paper is to replace g(s; M1 ; M2 ; : : : ; MN ), which is a known nonlinear function involving integrals, with g ?(s; M1 ; M2 ; : : : ; MN ), which is a linear combination of standard sigmoid functions by using the continuous and nonlinear mapping of the inputs to the outputs inherent in NNs.

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352 Table 1 Speed comparison of backpropagation algorithms Training optimization methods Gradient descent (GDBP) Quasi-Newton (QNBP) Levenberg–Marquardt (LMBP)

345

Training CPU times (s) 2467 583 356

The NN: 3 input nodes, 1 single 12-neuron hidden layer, 1 output neuron, and the performance gradient is 1e-4.

The NN in this paper is designed and implemented by using MATLABJ M-scripts. Levenberg– Marquardt backpropagation (LMBP) training algorithm [17,18] is adopted. Like the Quasi-Newton (or secant) methods, the LMBP algorithm can approach second-order training speed without having to compute the Hessian matrix [19]. When the performance function has the form of a sum of squares (as is typical in training feedforward NNs), the Hessian matrix H can be approximated as a H = J T J and the gradient can be computed as g = J T e, where J is the Jacobian matrix that contains the ?rst derivatives of the NN errors with respect to the weights and biases and e is the vector of NN errors. The Jacobian matrix can be computed through a standard backpropagation technique that is much less complex than computing the Hessian matrix. The update scheme of the LMBP algorithm is thus as: Xk +1 = Xk ? [ J T J + I ]?1 J T e; where Xk is the vector of weights and biases at the k th iteration. When the scalar is zero, this is exactly the Newton’s optimization, using the approximate Hessian matrix. While is large, this becomes the gradient descent method with a small step size. The Newton’s method is faster and more accurate near the error minimum. The aim is to switch to the Newton’s method as quickly as possible. Thus, is decreased after each successful step (reduction in performance function) and is increased only when a tentative step would increase the performance function. In this way, the performance function will always be reduced at each iteration of the algorithm. Another reason to use the LMBP algorithm is that it is much faster than other backpropagation algorithms as shown in Table 1 for Example 1. Training and testing (validation) data sets are generated as follows: ?rst, a suitable number of input vectors, (s; M1 ; M2 ; : : : ; MN ), are chosen or generated randomly from the expected value ranges of s; M1 ; M2 ; : : : ; and MN ; next, the input vectors are normalized and for each input vector, the desired output, g(s; M1 ; M2 ; : : : ; MN ), is calculated with Eq. (7). The training and testing data sets consist of di erent pairs of the input vector and its corresponding output. An important but hard problem in NN design and implementation is to determine the number of hidden units. The di culties root from the so-called over?tting and under?tting. The optimal number of hidden nodes depends in a complex way on the complexity of the function to be approximated, the type of activation functions in hidden layers, the numbers of input and output nodes; the training algorithm, the size of training data set, the amount of noise in training data and so forth. This topic remains an art and no once-for-all rules are available to follow. Sarle [20], Weigend [21] and Tetko et al. [22] report that there seems to be no upper limit on the number of hidden units, other than that imposed by computer time and memory requirements. In the LMBP algorithm used in the paper,

346

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

validation vectors (the elements of which are the global parameters such as the maximum number of epochs, the maximum amount of time, performance goal, and the minimum performance gradient, etc.) are used to stop training early if the NN performance on the validation vectors fails to improve or remains the same. Thus a relatively large number of hidden units can be used without bringing in over?tting. Hence, both under?tting and over?tting on NN training can be avoided e ectively to a certain degree. In addition, test vectors are used as a further check that the NN is generalizing well. 4. Examples 4.1. Example 1 In order to visualize the comparison between the analytical and the NN solutions (as any entity beyond 3 dimensions is unable to be visualized e ectively), we ?rst consider the design problem of a 2-stage, analytically solvable parallel–series system in which N = 2; C1i = 15; C2i = 20; CT = 150; (s) = 10s; f1i = 1 and f2i = 2s. Case 1: Analytical solution. The objective function U (M1 ; M2 ) is derived as U (M1 ; M2 ) = 10 M2 2M1 + M2 2M1 + ? 2M1 + 1 M2 + 1 2M1 + M2 + 1 (10)

and the solution that maximizes this objective function subject to the resource constraint is: M1 = 3; M2 = 6; U (3; 6) = 7:9121: Case 2: NN approximation. The training and validation data sets are obtained as follows: initial input vectors, [s; M1 ; M2 ]T , are generated randomly in the expected value ranges (s ∈ [0; 1]; M1 ; M2 ∈ [0; 10]) and normalized to be y = [s; m1 ; m2 ]T ∈ [0; 1]3 ; the desired output target g(s; M1 ; M2 ) is calculated according to the following equation: ? ?

2

g(s; M1 ; M2 ) = Pr(’(x) ? s) =

i=1

?1 ?

Mi

s

j =1

0

fij (t ) d t ? = (1 ? s)M1 (1 ? s2 )M2 :

? (M1 ; M2 ), constructed from the approximate system distriThe approximate objective function, U bution function, g ?(s; M1 ; M2 ), with Eq. (8), is ? (M1 ; M2 ) = 10 U

c i+1 i=1

1?

1 1 Ai + ln Ai wi 2 Bi

;

(11)

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

Objective function for the actual model

347

10 9 System utility U (M1,M2) 8 7 6 5 4 3 2 1 10

8 6 4 M2 2 0 0 4 2 6 M1 8

10

Fig. 3. U1 (M1 ; M2 ).

where Ai = 1 + e?(wi1 +wi2 +wi3 m1 +wi4 m2 ) ; Bi = 1 + e?(wi1 +wi3 m1 +wi4 m2 ) : The training data set contains 240 pairs of input vector and its corresponding output. The objective of training is that the percent error is smaller than 1%. We achieved this goal when the number of the hidden neurons c is equal to 12 after 2500 training epochs. The actual percent error is 0.80477%. The maximum error for a randomly generated testing set consisting of 50-pair data is 2.0628%. The ?nal optimal design solution is M1 = 3; M2 = 6; ? (3; 6) = 7:9210: U Although the analytical expression of the actual objective function in Eq. (10) obtained analytically and the approximate one in Eq. (11) obtained using the NN approximation are completely di erent, ? (M1 ; M2 ) shown in Figs. 3 and the actual objective function U (M1 ; M2 ) and the approximate one U 4 respectively are almost identical. The ?nal optimal solutions for both methods are the same, i.e., M1 = 3; M2 = 6. The values of the true objective function and the approximate objective function are ? (3; 6) = 7:9210, In addition, from Fig. 5, the maximum percent very close, i.e., U (3; 6) = 7:9121; U error between these two functions is under 3.5%. 4.2. Example 2 In this example, we use a 4-stage, analytically unsolvable parallel–series system in which N = 4; (s) = 10s; C1i = 3200; C2i = 1700; C3i = 830; C4i = 2500; and CT = 160; 000. The probability

348

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

Objective function for the approximated model

10 9 System utility U(M1,M2) 8 7 6 5 4 3 2 1 10 8 6 4 M2 2 0 0 2 6 4 M1 8 10

? 1 (M1 ; M2 ). Fig. 4. U

Approximation percent error for objective function %

10 8 percent error % 6 4 2 0 10 8 6 M2 4 2 0 0 4 2 M1 6 8 10

? (M1 ; M2 ) and U1 (M1 ; M2 ). Fig. 5. Percent error between U

density functions of components in the four subsystems are three commonly used distributions: unit distribution, triangular distribution and Beta distribution. f1i = 1; unit distribution; f2i = 2s; triangular distribution; f3i (s) = f4i (s) = ( + ?) s ( ) (?) ( + ?) s ( ) (?)

?1

(1 ? s)??1 ; beta distribution where (1 ? s)??1 ; beta distribution where

= 2; ? = 3:5; = 5; ? = 2:

?1

The density functions of the components in the 4 subsystems are plotted in Fig. 6.

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

2.5 f1i f2i f3i f4i

349

2

1.5

1

0.5

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 6. Component state distribution functions for Example 2.

The desired output targets for NN training, i.e., actual system distribution function, are calculated with ? ?

4

g(s; M1 ; M2 ; M3 ; M4 ) =

i=1

?1 ?

Mi

s

j =1

0

fij (t ) d t ?

= (1 ? s)M 1 (1 ? s2 )M 2 (1 ? (3:5(1 ? s4:5 ) ? 4:5(1 ? s3:5 ) + 0:0635)M 3 (1 ? 6s5 + 5s6 )M 4 : (12)

We can see that although Eq. (12) is also complicated, it is calculable. But the objective function U (M1 ; M2 ; M3 ; M4 ) constructed from Eq. (6) is too complex and seems impossible to calculate analytically so that the whole optimization design is unsolvable analytically. On the contrary, it is straightforward to solve this optimization problem using the NN approximation because the expression format of the approximate objective function is the same as in Example a 1. In other words, the approximate objective function, U (M1 ; M2 ; M3 ; M4 ), constructed from the approximate system distribution function is ? (M1 ; M2 ; M3 ; M4 ) = 10 U where Ai = 1 + e?(wi1 +wi2 +wi3 m1 +wi4 m2 +wi5 m3 +wi6 m4 ) ; Bi = 1 + e?(wi1 +wi3 m1 +wi4 m2 +wi5 m3 +wi6 m4 ) :

L i+1 i=1

1?

1 1 Ai + ln Ai wi 2 Bi

;

350

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

Fig. 7. Training performance versus training epoch for Example 2.

The training data set containing 1600 training pairs is generated randomly. The size of the hidden layer c = 19. After 2500 epochs of training iterations, the actual percent training error is 0.9859%. The maximum validation error is 2.4532%. Fig. 7 shows the training performance versus training epochs. After an exhaustive search, the ?nal optimal solution is M1 = 14; M2 = 9; M3 = 36; M4 = 15 and the corresponding approximate utility is U (12; 7; 35; 32) = 8:4384. 5. Conclusion and discussions The continuous-state model for parallel–series system optimal design makes sense in both theory and application. The advantage of a continuous-state model is that it models realities more accurately. However, in some cases, it is very di cult to solve the continuous-state model optimal design problem using classical optimization algorithms because the objective function becomes quite complex. There are also situations wherein explicit system objective function is not available except as a set of experimental data. To overcome these di culties, we have proposed to use the NN approximation to approximate the objective functions.

a

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

351

Compared to classical direct search optimization methods, another advantage of the NN approximation is that no matter how complicated the system may be, the form of the approximate objective functions is always the same. Speci?cally, it is always a linear combination of some kind of nonlinear functions, for example, sigmoidal functions in this paper. Once the approximate model is available, the subsequent search method and procedure may be identical. This is especially useful and e cient for the cases when the system is large, the state distribution functions of the components in the same subsystem are not identical, and= or there is no explicit system model available except an experimental data set.

References

[1] Kuo W, Prasad VR, Tillman FA, Huang CL. Optimal reliability design. New York: Cambridge University Press, 2001. [2] Misra KB, Sharma U. An e cient approach for multiple criteria redundancy optimization problems. Microelectronics and Reliability 1991;31(2):303–21. [3] Gen M, Ida K, Lee JU. A computational algorithm for solving 0-1 goal programming with GUB structures and its applications for optimization problems in system reliability. Electronics and Communication in Japan: Part 3 1990;73:88–96. [4] Gen M, Ida K, Tsujimura Y, Kim CE. Large scale 0-1 fuzzy goal programming and its application to reliability optimization problem. Computers and Industrial Engineering 1993;24:539–49. [5] Coit DW, Smith AE. Reliability optimization of the parallel–series systems using a genetic algorithm. IEEE transactions on Reliability 1996;45(2):254–60. [6] Monga A, Zuo MJ. Optimal system design considering maintenance and warranty. Computers and Operations Research 1998;25(9):691–705. [7] Barlow RE, Wu AS. Coherent systems with multi-state components. Mathematics of Operations Research 1978;3(4):275–81. [8] Gri th WS. Multi-state reliability models. Journal of Applied Probability 1980;17:735–44. [9] Xue J, Yang K. Dynamic reliability analysis of coherent multi-state systems. IEEE Transactions on Reliability 1995;R-44(4):683–8. [10] Huang J, Zuo MJ, Wu Y. Generalized multi-state k-out-of-n:G systems. IEEE Transactions on Reliability 2000;49(1):105–11. [11] Zuo MJ, Choy LF, Yam RCM. A model for optimal design of multi-state parallel–series systems. 1999 IEEE Canadian Conference on Electrical and Computer Engineering, Edmonton, May 9 –12, 1999. p. 1770 –3. [12] Ross SM. Multivalued state component systems. The Annals of Probability 1979;7(2):379–83. [13] Block HW, Savits TH. A decomposition for multi-state monotone systems. Journal of Applied Probability 1982;19:391–402. [14] Cappelle B, Kerre E. Computer assisted reliability analysis: an application of possibilistic reliability theory to a subsystem of a nuclear power plant. Fuzzy Sets and Systems 1995;74:103–13. [15] Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 1989;2(4):303–14. [16] Aourid M, Do XD. NN approximation: application to peak switching over-voltage determination in power systems. IEEE International Conference on NNs Conference Proceedings 1995;1:200–4. [17] Mathworks, MATLAB, High-Performance Numeric Computation And Visualization Software: Reference Guide. Natick, MA, USA, 1992. [18] MATLABJ Version 6.0.0.88 Release 12, September 22, 2000, The MathWorks, Inc. [19] Scales LE. Introduction to non-linear optimization. New York: Springer, 1985. [20] Sarle WS. Stopped training and other remedies for over?tting. Proceedings of the 27th Symposium on the Interface of Computing Science and Statistics 1995. p. 352– 60.

352

P.X. Liu et al. / Computers & Operations Research 30 (2003) 339 – 352

[21] Weigend A. On over?tting and the e ective number of hidden units. Proceedings of the 1993 Connectionist Models Summer School, 1994. p. 335 – 42. [22] Tetko IV, Livingstone DJ, Luik AI. Neural network studies. 1. Comparison of over?tting and overtraining. J. Chem. Info. Comp. Sci. 1995;35:826–33. Peter X. Liu received the B.S. and M.S. degree in mechanical engineering from Northern Jiaotong University, Beijing, China. He is currently pursuing the Ph.D. degree at the Advanced Robotics and Teleoperation (ART) Laboratory, University of Alberta. His research interests include real-time data transmission in the Internet, Internet based teleoperation, softHaptics, and mobile robotics. Ming Jian Zuo is Professor in the Department of Mechanical Engineering at the University of Alberta in Canada. He received a B.Sc. degree in Agricultural Engineering from Shandong Institute of Technology, China and M.Sc. and Ph.D. degrees in Industrial Engineering from Iowa State University, USA. His research interests include system reliability analysis, condition based maintenance modeling, and manufacturing systems. Max Meng received his Ph.D. degree in Electrical and Computer Engineering from the University of Victoria in Canada. He is currently a Professor of Electrical and Computer Engineering and the Director of the Advanced Robotics and Teleoperation (ART) Laboratory at the University of Alberta in Canada. His research expertise is in the area of Robotics, Network Enabled Services, Intelligent and Adaptive Systems, and Human–Machine Interface, with medical, industrial, and military applications. He has published over 100 journal and conference papers. His research team invented the world’s ?rst naturally moving prosthetic “robotic eye” actively controlled using human EOG signals. He is responsible for the introduction of the concept of “SoftHaptics”. He is an editor of the IEEE= ASME Transactions on Mechatronics.

- Neural network design for J function approximation in dynamic programming
- A Wavelet Neural Network for Function Approximation and Network Optimization,” Intelligent
- A Neural Network Based Design of Optimal Controllers
- Identification of optimal operating point of PV modules using neural network for real time maximum p
- OPTIMAL DESIGN OF WATER DISTRIBUTION NETWORK USING SHUFFLED COMPLEX EVOLUTION
- Optimal Design of Linear Network Coding for Information Theoretically Secure Unicast
- Bayesian Regularization in a Neural Network Model to Estimate Lines of Code Using Function
- Neural network exploration using optimal experiment design
- On Neural Network Topology Design for Nonlinear Control
- Face Detection using a Modified Radial Basis Function Neural Network

更多相关文章：
**
关于《On the ***Approximation* *of* Differentiable *Functions* by ....pdf

关于《On the*Approximation* *of* Differentiable *Functions* by Euler Means》一文的注记_电子/电路_工程科技_专业资料。指出了文献[1]中引理2不成立,并建立了新的...**
***Neural* *networks* *for* *of* continuous *optimal* *approximation* ....pdf

*Neural* *networks* *for* *of* continuous *optimal* *approximation* *functions* in R d_专业资料。Ap.Matp1h.J.Chneeis Uni v 21,73:34 022()3534 Nerlnewoksfroia ...**
...from Feedforward ***Neural* *Network* *for* *Function* *Approximation*....pdf

Rule Extraction from Feedforward*Neural* *Network* *for* *Function* *Approximation*_专业资料。The paper presents a method *of* rule-*based* interpretation *of* feedforward ...**
A Wavelet ***Neural* *Network* *for* *Function* *Approximation* and ....pdf

In the first process, the*network* structure is determined by *using* wavelet analy A Wavelet *Neural* *Network* *for* *Function* *Approximation* and *Network* Optimization ...**
***Function* *Approximation* by Polynomial Wavelets Generated from_....pdf

Keywords :*Neural* *Networks*, *Function* *Approximation*, Wavelets. 1 Introduction ...One *of* the advantages *of* *using* wavelets in the *design* *of* *neural* *networks* ...**
***Function* *approximation* from noisy data by an incremental RBF ....pdf

Assuming the data a ected by noise, an*optimal* *network* *approximation* must ...*Function* *Neural* *Networks*, *Neural* Nets WIRN 98, M.Marinaro and R.Taglia...**
***Using* the wavenet *for* *function* *approximation*.pdf

*Using* the wavenet *for* *function* *approximation*_专业资料。When the aim is to make an arbitrary nonlinear mapping, *neural* *networks* are known to be a suitable...**
...***using* Phi-machines and constructive *function* *approximation*....pdf

machines and constructive*function* *approximation*_专业...(1997). Constructive *neural* *network* learning ...& Rousu, J. (1996). Finding *optimal* multi- ...**
...Tile Coding ***for* Value *Function* *Approximation*_免....pdf

Tile Coding*for* Value *Function* *Approximation*...The agent strives to learn the *optimal* value ...*neural* *network* or the resolution *of* a state ...**
...incompatability ***of* *function* *approximation* and factored ....pdf

*function* *approximation* and factored dynam_专业资料。...is generated; *based* on ×? and ? , ? ...which even *approximate* the *optimum* return to ...**
Efficient Value-***Function* *Approximation* via Online Linear ....pdf

Efficient Value-*Function* *Approximation* via Online Linear Regression - One *of* the key problems in ...... compact *function* *approximation* has to be *used*. ......**
Abaqus_2003.1_图文.ppt
**

*Optimum* *Design* Quality Efforts Made Quality Efforts...Distribution *of* a *function* vs. the random ...Low level DOE can be *used* *Approximation* Run ...**
...and ***approximate* convexity *of* Takagi-type *functions*.pdf

(2) Theorem 1 is*used* to prove the following result, which specializes ...The relationship between the Takagi *function* and the binary digital sum S2 ...**
***Approximation* *of* multivariate *functions*.pdf

K. and Xin Li,*Approximation* by ridge *functions* and *neural* *networks* with ...Shepp, *Optimal* reconstruction *of* a *function* from its projections, Duke Math....**
Gas content prediction ***based* on GA-RBF *neural* *network*.pdf

*based* on GA-RBF *neural* *network* optimized by ...*function* *approximation*, classification and learning ...*optimal* solution *of* the problem, as follows: T ...**
***Approximation* *of* Elementary *Functions* *Using* a Weighted Sum *of*....pdf

*function* *approximation* *using* a weighted sum *of* bit...(7) are *optimal* when all bit-products are ..."Hardware *designs* *for* exactly [3] R.-C. Li,...**
Boundary conditions in the envelope ***function* *approximation* as....pdf

M.-E. Pistol 1 Boundary conditions in the envelope*function* *approximation* as applied to semiconductor heterostructures: the multi-band case M.-E. Pistol ...**
***Approximation* by polynomials in a weighted space *of* ....pdf

*Approximation* by polynomials in a weighted space ...We consider a *function* χ∈ C ∞ (R) such .... *Using* Theorem 1 and results *of* [5] (in ...**
ONESIDED ***APPROXIMATION* BY ENTIRE *FUNCTIONS*.pdf

(E ? (δ, f )p is de?ned*using* the reverse inequality.) In previous...xn +. 3. Entire *approximation* n We de?ne the *function* xn ? to be x...**
A new ***approximation* method *of* the quadratic discriminant ....pdf

A new*approximation* method *of* the quadratic ...*function* *of* multivariate normal distribution is *used*...cant and it is di?cult to ?nd *optimal* ... 更多相关标签：

关于《On the

Rule Extraction from Feedforward

In the first process, the

Keywords :

Assuming the data a ected by noise, an

machines and constructive

Tile Coding

Efficient Value-

(2) Theorem 1 is

K. and Xin Li,

M.-E. Pistol 1 Boundary conditions in the envelope

(E ? (δ, f )p is de?ned

A new