...

International Electrical Engineering Journal (IEEJ) Vol. 6 (2015) No.3, pp. 1803-1808

by user

on
1

views

Report

Comments

Transcript

International Electrical Engineering Journal (IEEJ) Vol. 6 (2015) No.3, pp. 1803-1808
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/
Support Vector Machines and Artificial
Neural Networks for Identification of
Residence Time Distribution Signals
H. Kasban1, H. Arafa1 , S. M. S. Elaraby1 , O. Zahran2, M. El-Kordy2 and F. E. Abd El-Samie2
1
Nuclear Research Center, Atomic Energy Authority, Egypt
2
Faculty of Electronic Engineering, Menofia University, Egypt.
E-mails : [email protected] , [email protected]
Abstract— This paper presents a practical comparison between
the Support Vector Machines (SVMs) and the Artificial Neural
Networks (ANNs) as identifiers for the Residence Time
Distribution (RTD) signal identification. In these identifiers, the
cepstral features are extracted from the signal or from its
power density spectrum (PDS) estimated using eigenvector
method, or from the Discrete Cosine Transform (DCT), then
the extracted features feed the identifiers. Both identifiers have
been tested using the same RTD signals. The performance of
these identifiers is evaluated in the presence of different types
of noise. The simulation results proved that, the ANNs based
identifier is more reliable in RTD signal identification, but it
takes more time with respect to the SVMs based identifier.
Index Terms— Support Vector Machines, Artificial Neural
Networks, Residence Time Distribution
I. INTRODUCTION
RTD is the total time spent by the particles in the system. RTD
measurement is used for determining the possible system
malfunctions such as channeling, bypassing, short-circuiting
and existence of dead volumes in many industrial systems
[1-3]. Also, it can be used for optimizing the design of the
industrial system at the design stage. RTD measurement can
be performed by the injection of a suitable tracer into the
system and monitoring the concentration of the tracer using
radiation detectors placed at one or more locations [4-6]. The
main advantages of using radiotracer in RTD measurement
are; physico-chemical compatibility, high detection
sensitivity, on-line detection and the availability of a number
of radiotracers for different phases and their limited memory
effects.
The main problems of using these techniques are the difficulty
of identification of the obtained signals and the requirement
of skilled experts in the identification process of the output
signal. Normally, the identification of the output signal is
performed manually, depending heavily on the skills and the
experience of a trained operator. This process is time
consuming and the results typically suffer from inconsistency
and errors. Also, the RTD signal may be subject to different
sorts of noise. This leads to errors in the RTD calculations,
and hence leads to wrong analysis in the determination of
system malfunctions. To overcome these problems, some
techniques have been presented for treatment and
identification of the RTD signal. RTD signal processing has
been presented before using Z and Fast Fourier transforms
and some digital signal processing techniques [7, 8].
Kasban et. al. presented an approach for RTD signal
identification based on transfer domains [9, 10]. Discrete
wavelet Transform (DWT), DCT and Discrete Sine
Transform (DST) have been tested and compared. The
Cepstral features are extracted from the signal or from one of
its domains transforms, then the neural networks are used for
matching the extracted features of the original RTD signal in
the presence of noise. The experimental results show that the
highest identification rate is obtained when the features are
extracted from the DCT of the RTD [9, 10]. Zahran et. al.
used the PDS estimated using its different estimation methods
instead of using transforms domain in [9]. The identification
results are compared to different estimation methods in order
to select the best PDS estimation method for RTD signal
identification. Neural networks are used for training and
testing in the proposed approach. The experimental results
showed that; the proposed approach with features extracted
from the PDS of the RTD signals estimated using eigenvector
method provides the highest identification rate [11, 12].
References 9 -12 are used only ANNs for identification with
different features and transforms, although they achieved
good identification rate, we are looking for higher
identification rate within a shorter time. This paper presents a
comparison between using the SVMs and ANNs as identifiers
for the RTD signal identification purpose. In this paper, the
cepstral features [9-12] are extracted from the signal or from
the PDS estimated using eigenvector methods (best results in
[11, 12]) or from the DCT of the signal (best results in [9,
10]), then the extracted features feed the identifiers. The rest
of this paper is organized as follows: Section 2 presents a
dissection about using the SVMs and ANNs in the
identification process. Section 3 presents the paper
methodology. Section 4 presents the experimental results and
discussions. Finally, Section 5 gives the concluding remarks.
1803
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/
II. SVMS VERSUS ANNS
SVMs are developed by Vapnik and colleagues at Bell
laboratories [13]. SVMs have been used by many researchers
in different applications [14-17]. SVMs considered a set of
related supervised learning techniques, it belongs to a family
of generalized linear identification. SVMs are used for
minimizing the empirical identification error and maximizing
the geometric margin. The support-vector network is a
learning machine for two-group classification problems. The
basic SVM deals with two-class problems in which the data
are separated by a hyperplane defined by a number of support
vectors. It operates by mapping the data of interest to a high
dimensional space and generating a separating hyperplane in
that space. The high dimensional separating hyperplane can
be used for hypothesis testing.
The identification using SVMs composed of two processes:
building a model that simulates the identification system and a
feature matching process that evaluates the performance of
the model by using a test set of signals. In the modeling step,
the RTD signals are stored in the system using features that
are extracted during the training phase. When an unknown set
of signal arrives, a feature matching technique is applied to
map the features from this set in the model. SVMs minimize
the empirical identification error and maximize the geometric
margin. SVMs map an input vector to a higher dimensional
space, where a maximal separating hyperplane is constructed.
Two parallel hyperplanes are constructed on each side of the
hyperplane that separates the data. The separating hyperplane
is the hyperplane that maximizes the distance between the two
parallel hyperplanes.
On the other hand, the history of neural networking, arguably
started in the late 1800s with scientific attempts to study the
workings of the human brain. In 1890, William James
published the first work about brain activity patterns. The first
artificial neuron was produced in 1943 by neurophysiologist
Warren McCulloch and the logician Walter [18]. ANNs have
been used by many researchers in different applications
[19-23]. ANNs, like people, learn by examples, learning in
biological systems involves adjustments to the synaptic
connections that exist between the neurons. Artificial neurons
are simulations of biological neurons. They receive one or
more input and sum them to produce an output. ANNs
composed of a large number of highly interconnected
processing elements (neurons) working in unison to solve
specific problems.
The identification using ANNs composed of two phases;
training and testing. The training of neural network is
accomplished by adjusting its weights using a training
algorithm. The training algorithm adapts the weights by
attempting to minimize the sum of the squared error between a
desired output and the actual output of the output neurons
given:
E
1 O
2
 Do  Yo 
2 o 1
(1)
where Do and Yo are the desired and actual outputs of the oth
output neuron, respectively, and O is the number of output
neurons. Each weight in the neural network is adjusted by
adding an increment to reduce E as rapidly as possible. The
adjustment is carried out over several training iterations, until
a satisfactorily small value of E is obtained or a given number
of epochs are reached.
The similarities between SVMs and ANNs are; SVM with
sigmoid kernel is equivalent two-layer feed forward ANNs
and SVMs with Gaussian kernel is equivalent radial basis
function network. Table (1) summarizes the advantages and
disadvantages of SVMs and ANNs.
Table (1): Advantages and disadvantages of SVMs and ANNs
Advantages


SVMs
Fewer parameters to consider (kernel, cost).
Works well with fewer training samples (number of
support vectors).


ANNs
An ability to learn how to do tasks based on the data
given for training or initial experience.
Can create its own organization or representation of
1804
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/






Disadvantage
s


Prediction accuracy is generally high
Fast evaluation of the learned target function
Effective in high dimensional spaces.
Uses a subset of training points in the decision
function so it is memory efficient.
Different Kernel functions can be specified for the
decision function.
Learning result is more robust, works when training
examples contain errors.
Over fitting is not common.
Problem need to be formulated as 2-class
classification.

Poor performances when the number of features is
greater than the number of samples.

Difficult to understand the learned function.




the information it receives during the learning time.
Computations may be carried out in parallel, and
special hardware devices are being designed and
manufactured taking advantage of this capability.
Fault tolerance via redundant information leads to a
corresponding degradation of performance.
Suffer from multiple local minima.
Computational complexity depends
dimensionality of the input space.
on
the
1805
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/
III. METHODOLOGY
The method of RTD signal identification is shown in
figure (1). The method has two phases; a training phase
and a testing phase. In the training phase, features are
extracted from each signal or from its PDS or from its
DCT. These features are used to train an SVMs or ANNs.
Features are extracted from every incoming signal in the
testing phase, and a decision is made through feature
matching and decision making step to give RTD signal or
not. The DCT expresses a sequence of finitely many data
points in terms of a sum of cosine functions oscillating at
different frequencies. The DCT of a sequence {x (n), 0 ≤ n
≤ N-1} is given by [24]:
N 1
 (2n  1)k
(2)
E(k)   (k ) E (n) cos(
)
2N
n0
where , 0 ≤ k ≤ N-1, and
 (0) 
1
N
,
 (k ) 
2
N
(3)
Power Density Spectrum (PDS) describes how
the power of a signal is distributed with frequency. There
are many methods for estimating the PDS, As eigenvector
method provided the best results in RTD identification, so
it will be used in this paper. This method is based on an
Eigen-analysis of the autocorrelation matrix of the
noise-corrupted signal. Eigen-analysis is used for
partitioning the Eigenvectors and the Eigenvalues of the
autocorrelation matrix of a noisy signal into two
subspaces; the signal subspace composed of the principle
Eigenvectors associated with the largest Eigenvalues and
the noise subspace represented by the smallest
Eigenvalues. The decomposition of a noisy signal into a
signal subspace and a noise subspace forms the basis of
the Eigen-analysis methods [25, 26]. This method can be
described in the following steps:
a) Estimate the autocorrelation matrix.
b) Estimate the Eigenvalues of the autocorrelation
matrix αk.
c) Estimate the corresponding Eigenvectors vk .
d) Estimate the PDS using the following equation
[12]:
(4)
P eig ( f ) 
1
P
  k e t ( f )v k
2
k  M 1
where ak are the weighting factors and vk, (k = M + 1
, . . . , p) are the noise subspace Eigenvectors.
Feature extraction means reducing the amount of
data present in the RTD signal while retaining the signal
discriminative information. In this paper the cepstral
features [9-12] are extracted from the signal or from its
power density spectrum (PDS) or from one of its domains
transforms, and then the extracted features feed the
identifiers. The SVMs and ANNs based identifiers have
been tested using the same RTD signals.
RTD signal
RTD signal
degraded RTD signal
PDS
DCT
.
Features extraction
ANNs
Features
database
PDS
DCT
Features extraction
ANNs
training
SVMs
training
SVMs
Features
database
ANNs
testing
SVMs
testing
Decision
Training phase
Testing phase
Figure (1): RTD signal identification method
1806
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/
IV. EXPERIMENTAL RESULTS
20 RTD signals are used in the training phase for
both identifiers. In the testing phase, the comparison has
been done for three signal degradations; Gaussian, Rayleigh
and Rician noise. For each time, the features are extracted
separately and compared from the signal, from the DCT of
the signal or from the PDS of the signal estimated by
eigenvector method. The performance evaluation metric in
the experiments is the identification rate:
Identification rate 
The number of success identifica tions
The total number of identifica tion trials
(5)
Table (2) shows the identification rate versus the signal to
noise ratio (SNR) when the RTD signals degraded by
Gaussian noise.
Tables (2): Identification rate (%) when the signal degraded by Gaussian
noise
SNR
(dB)
0
5
10
15
20
25
30
Signal
65
65
70
75
75
75
80
ANNs
DCT
90
90
95
95
95
100
100
PDS
90
95
95
95
100
100
100
Signal
40
50
60
65
70
75
80
SVMs
DCT
70
70
75
75
80
85
95
PDS
65
70
70
80
90
95
100
The results show that; the identification rate increase by
increasing the SNR, where it become 100% at 20 dB for
ANNs based identifier while the 100% identification rate
obtained at 30 dB when using the SVMs based identifier, it is
noted that, the ANNs based identifier give higher
identification rate than the SVMs based identifier, SVMs
based identifier is affected by SNR more than the ANNs
based identifier, and when the features are extracted from the
PDS of the signal, the highest identification rate is obtained.
Table (3) shows the identification rate versus noise
variance when the RTD signals degraded by Rayleigh noise.
Tables (3): Identification rate (%) when the signal degraded by Rayleigh
noise
Noise
variance
0
5
10
15
20
25
30
35
40
45
50
Signal
65
65
65
65
60
55
55
55
50
40
40
ANNs
DCT
100
95
90
85
85
85
85
80
70
70
65
PDS
100
95
95
95
90
90
90
90
85
85
85
Signal
65
65
65
65
65
60
60
60
55
55
50
SVMs
DCT
95
95
85
85
85
85
85
80
75
75
70
PDS
100
95
95
90
85
80
80
80
80
80
80
The results show that; the identification rate decreases with
increasing the noise variance. It is noted that, the ANNs based
identifier gives a higher identification rate than the SVMs
based identifier, and the highest identification rate is obtained
when the features are extracted from the PDS of the signal.
Table (4) shows the identification rate versus Rician
probability density when the signal degraded by Rician
noise.
Tables (4): Identification rate (%) when the signal degraded by Rician noise
Probability
density
0
5
10
15
20
25
30
35
40
45
50
Signal
80
60
45
35
20
20
20
15
15
15
10
ANNs
DCT
100
80
75
70
65
70
65
60
55
45
30
PDS
100
75
75
75
75
75
75
75
75
75
75
Signal
75
70
55
40
25
20
20
20
20
10
10
SVMs
DCT
85
75
75
60
55
30
30
20
20
10
10
PDS
100
75
75
75
75
70
70
65
65
60
60
The results show that; the identification rate decreases with
increasing the noise probability density. The results proved
the superiority of the ANNs based identifier to the SVMs
based identifier.
To compare between the SVMs and ANNs based
identifiers from the point of execution time, table (5) shows
the execution time (Sec) of the identification process for
different signal degradations. This time is the CPU processing
time using DELL laptop with Intel core i5 CPU and 4 GB
RAM running with MATLAB 7.6. It is found that the ANNs
based identifier takes more time with respect to SVMs based
identifier.
Tables (5): Testing time (Sec) of identification for different signal
degradations
ANNs
SVMs
Noise
Signa
DCT PDS Signal
DCT
PDS
l
Gaussian
24
31
32
19
20
23
Rayleigh
21
33
31
11
18
30
Rician
21
35
40
21
21
23
V. CONCLUSION
The paper presented a practical comparison between the
SVMs and ANNs as identifiers for RTD signal identification.
The previous research in the RTD identification used only the
ANNs with different features and transforms, although they
achieved good identification rate, we are looking for higher
identification rate within a shorter time. In this paper, the
cepstral features are extracted from the signal or from the PDS
estimated using eigenvector method or from the DCT of the
signal, then the extracted features feed the identifiers. The two
identifiers have been tested using the same RTD signals. The
performance of these identifiers is evaluated in the presence
1807
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
International Electrical Engineering Journal (IEEJ)
Vol. 6 (2015) No.3, pp. 1803-1808
ISSN 2078-2365
http://www.ieejournal.com/
of different types of noise. The simulation results proved that,
the ANNs based identifier give higher identification rate than
the SVMs based identifier, and the highest identification rate
is obtained when the features are extracted from the PDS of
the signal. In the future other techniques can be tested for
enhanced the identification process such as; hidden Markov
model or Fuzzy algorithms.
REFERENCES
[1] International Atomic Energy Agency, “Radiotracer applications in
industry: a guidebook”, Safety Reports Series 423, IAEA, Vienna
Austria, 2004.
[2] International Atomic Energy Agency, “Diagnosis of industrial
reactors by radiotracers: RTD applications”, Technical Report Series
423, IAEA, Austria, 2005.
[3] International Atomic Energy Agency, “Residence time distribution
method for industrial and environmental applications”, Training
Course Series 31, IAEA, Vienna Austria, 2008.
[4] H. Kasban, O. Zahran, H. Arafa, M. El-Kordy, Sayed M. S. Elaraby
and F. E. Abd El-Samie, “Laboratory experiments and modeling for
industrial radiotracer applications”, Applied Radiation and Isotopes,
Vol. 68, pp. 1045-1054, 2010.
[5] O. Zahran, H. Kasban, Horya A. Arafa, M. El-Kordy, and F. E. Abd
El-Samie, “residence time distribution measurements in phosphate
production unit using radiotracer techniques”, 48th Annual British
Conference on Non-Destructive Testing, Blackpool, UK, September
2009.
[6] H. Kasban, O. Zahran, Sayed M. S. Elaraby, S. EL-Rabaie, M.
El-Kordy and F. E. Abd El-Samie, “using radiotracer in industrial
applications”, 27th National Radio Science Conference, Menouf,
Egypt, March 2010.
[7] L. Furman, “Z-transform and adaptive signal processing in analysis of
tracer data”, Canadian Journal of Chemical Engineering, Vol. 80, No.
3, pp. 472-477, 2002.
[8] P. Viitanen, “Experiences on fast fourier transform as a deconvolution
technique in determination of process equipment residence time
distribution”, Applied Radiation and Isotopes, Elsevier, Vol. 48, No.
7, pp. 893-898, 1997.
[9] H. Kasban, O. Zahran, H. Arafa, M. El-Kordy, Sayed M. S. Elaraby
and F. E. Abd El-Samie, “An efficient approach for residence time
distribution signal processing and identification”, 7th International
Conference on Informatics and systems (INFOS 2010), Cairo, Egypt,
March 2010.
[10] H. Kasban, O. Zahran and F. E. Abd El-Samie, “New trends for on-line
troubleshooting in industrial problems using radioisotopes”, Journal
on Electronics and Electrical Engineering, Vol. 2, No. 3, pp.
284-292, 2010.
[11] H. Kasban, O. Zahran, H. Arafa, M. El-Kordy, Sayed M. S. Elaraby
and F. E. Abd El-Samie, “RTD signal identification using linear and
nonlinear modified periodograms”, International Conference on
Computer Engineering & Systems (ICCES'2011), IEEE, Cairo,
Egypt, November 2011.
[12] O. Zahran, H. Kasban, F. E. AbdEl-Samie and M. El-Kordy, “Power
density spectrum for the identification of residence time distribution
signals”, Applied Radiation and Isotopes, Elsevier, Vol. 70, pp.
2638–2645, 2012.
[13] C. Cortes and V. Vapnik, “Support-vector networks”. Machine
Leaming, Vol. 20, pp. 273-297, 1995.
[14] V. N. Vapnik, "An overview of statistical learning theory", IEEE
Transactions on Neural Networks, Vol. 10, pp. 988–1000, 1999.
[15] J. Weston and C. Watkins, “Support vector machines for multiclass
recognition”, ESANN’99, Bruges, Belgium, pp. 219–224, 1999.
[16] I. Guyon and N. Christianini, “Survey of support vector machine
applications”, NIPS’99 Special Workshop on Learning with Support
Vector, 1999.
[17] B. Scholkopf, “SVMs: A practical consequence of learning theory”,
IEEE Intelligent Systems, Vol. 13, pp.18–19, 1998.
[18] W. S. McCulloch and W.H. Pitts, “A logical calculus of the ideas
immanent in nervous activity”, Bulletin of Mathematical Biophysics,
Vol. 5, pp.115-137, 1943.
[19] T. Boukra, A. Lebaroud and G. Clerc, “Statistical and neural-network
approaches for the classification of induction machine faults using
the ambiguity plane representation” , IEEE Transactions on Industrial
Electronics, Vol. 60, No. 9, pp. 4034 - 4042, 2013.
[20] O. S. Stamenković, K. Rajković, A. V. Veličković, P. S. Milić, and V.
B. Veljković “Optimization of base-catalyzed ethanolysis of
sunflower oil by regression and artificial neural network models”,
Fuel Processing Technology, Vol. 114, pp. 101–108, 2013.
[21] L. H. Hassan, M. Moghavvemi, H. A. F. Almurib and O. Steinmayer
“Current state of neural networks applications in power system
monitoring and control”, Electrical Power and Energy Systems, Vol.
51, pp. 134-144, 2013.
[22] J. S. Torrecilla, C. Tortuero, J. Cancilla and P.-Rodríguez. “Estimation
with neural networks of the water content in imidazolium-based ionic
liquids using their experimental density and viscosity values”,
Talanta, Vol. 113, 93–98, 2013.
[23] D. Bibicu and L.Moraru, “Cardiac cycle phase estimation in 2-D
echocardiographic images using an artificial neural network”, IEEE
Transactions On Biomedical Engineering, Vol. 60, No. 5, pp. 1273 –
1279, 2013.
[24] R. Muralishankar, “Theoretical complex cepstrum of DCT and
warped DCT filters”, IEEE Signal Processing Letters, Vol. 14, No. 5,
pp. 367- 370, 2007.
[25] S. T. Ahsan, “Optimal window design, comparing it with the kaiser
window”, IEEE Potentials, pp. 40-44, 2002.
[26] D. G. Manolakis, V. K. Ingle, and S. M. Kogon, “Statistical and
adaptive
signal
processing”,
Artech
House,
ISBN
978-1-58053-610-3, 2005.
1808
Kasban et. al.,
Support Vector Machines and Artificial Neural Networks for Identification of Residence Time Distribution Signals
Fly UP