Research Papers

Robust Estimation of Balanced Simplicity-Accuracy Neural Networks-Based Models

[+] Author and Article Information
Hector M. Romero Ugalde

Université de Rennes 1,
Rennes F-35000, France
e-mail: hector.m.romero.ugalde@gmail.com

Christophe Corbier

Network and Telecommunications,
Université de Lyon,
Saint Etienne F-42023, France;
Network and Telecommunications,
Université de Saint Etienne,
Jean Monnet,
Saint-Etienne F-42000, France;
IUT de Roanne F-42334, France

1Corresponding author.

Contributed by the Dynamic Systems Division of ASME for publication in the JOURNAL OF DYNAMIC SYSTEMS, MEASUREMENT, AND CONTROL. Manuscript received September 2, 2015; final manuscript received January 17, 2016; published online March 3, 2016. Assoc. Editor: Ryozo Nagamune.

J. Dyn. Sys., Meas., Control 138(5), 051001 (Mar 03, 2016) (8 pages) Paper No: DS-15-1413; doi: 10.1115/1.4032687 History: Received September 02, 2015; Revised January 17, 2016

Neural networks are powerful tools for black box system identification. However, their main drawback is the large number of parameters usually required to deal with complex systems. Classically, the model's parameters minimize a L2-norm-based criterion. However, when using strongly corrupted data, namely, outliers, the L2-norm-based estimation algorithms become ineffective. In order to deal with outliers and the model's complexity, the main contribution of this paper is to propose a robust system identification methodology providing neuromodels with a convenient balance between simplicity and accuracy. The estimation robustness is ensured by means of the Huberian function. Simplicity and accuracy are achieved by a dedicated neural network design based on a recurrent three-layer architecture and an efficient model order reduction procedure proposed in a previous work (Romero-Ugalde et al., 2013, “Neural Network Design and Model Reduction Approach for Black Box Nonlinear System Identification With Reduced Number of Parameters,” Neurocomputing, 101, pp. 170–180). Validation is done using real data, measured on a piezoelectric actuator, containing strong natural outliers in the output data due to its microdisplacements. Comparisons with others black box system identification methods, including a previous work (Corbier and Carmona, 2015, “Extension of the Tuning Constant in the Huber's Function for Robust Modeling of Piezoelectric Systems,” Int. J. Adapt. Control Signal Process., 29(8), pp. 1008–1023) where a pseudolinear model was used to identify the same piezoelectric system, show the relevance of the proposed robust estimation method leading balanced simplicity-accuracy neuromodels.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.


Seo, J. , Khajepour, A. , and Huissoon, J. P. , 2011, “ Identification of Die Thermal Dynamics Using Neural Networks,” ASME J. Dyn. Syst., Meas., Control, 133(6), p. 061008. [CrossRef]
Han, X. , Xie, W. , Fu, Z. , and Luo, W. , 2011, “ Nonlinear Systems Identification Using Dynamic Multi-Time Scale Neural Networks,” Neurocomputing, 74(17), pp. 3428–3439. [CrossRef]
Jerome, T. C. , Martin, R. , and Atlas, L. , 1994, “ Recurrent Neural Networks and Robust Time Series Prediction,” IEEE Trans. Neural Networks, 5(2), pp. 240–254. [CrossRef]
Corbier, C. , and Carmona, J.-C. , 2015, “ Extension of the Tuning Constant in the Huber's Function for Robust Modeling of Piezoelectric Systems,” Int. J. Adapt. Control Signal Process., 29(8), pp. 1008–1023. [CrossRef]
Chen, D. S. , and Jain, R. C. , 1994, “ A Robust Backpropagation Learning Algorithm for Function Approximation,” IEEE Trans. Neural Networks, 5(3), pp. 467–479. [CrossRef]
Bloch, G. , Thomas, P. , and Theilliol, D. , 1997, “ Accommodation to Outliers in Identification of Non Linear SISO Systems With Neural Networks,” Neurocomputing, 14(1), pp. 85–99. [CrossRef]
Fu, Y.-Y. , Wu, C.-J. , Ko, C.-N. , and Jeng, J.-T. , 2011, “ Radial Basis Function Networks With Hybrid Learning for System Identification With Outliers,” Appl. Soft Comput., 11(3), pp. 3083–3092. [CrossRef]
Ko, C.-N. , 2012, “ Identification of Nonlinear Systems With Outliers Using Wavelet Neural Networks Based on Annealing Dynamical Learning Algorithm,” Eng. Appl. Artif. Intell., 25(3), pp. 533–543. [CrossRef]
Majhi, B. , and Panda, G. , 2011, “ Robust Identification of Nonlinear Complex Systems Using Low Complexity ANN and Particle Swarm Optimization Technique,” Expert Syst. Appl., 38(1), pp. 321–333. [CrossRef]
McKean, J. W. , 2004, “ Robust Analysis of Linear Models,” Stat. Sci., 19(4), pp. 562–570. [CrossRef]
Hsieh, J.-G. , Lin, Y.-L. , and Jeng, J.-H. , 2008, “ Preliminary Study on Wilcoxon Learning Machines,” IEEE Trans. Neural Networks, 19(2), pp. 201–211. [CrossRef]
Tsai, H.-H. , and Yu, P.-T. , 2000, “ On the Optimal Design of Fuzzy Neural Networks With Robust Learning for Function Approximation,” IEEE Trans. Syst., Man, Cybern., Part B, 30(1), pp. 217–223. [CrossRef]
Wang, W.-Y. , Lee, T.-T. , Liu, C.-L. , and Wang, C.-H. , 1997, “ Function Approximation Using Fuzzy Neural Networks With Robust Learning Algorithm,” IEEE Trans. Syst., Man, Cybern., Part B, 27(4), pp. 740–747. [CrossRef]
Rubio, J. J. , and Cruz, J. H. P. , 2014, “ Evolving Intelligent System for the Modelling of Nonlinear Systems With Dead-Zone Input,” Appl. Soft Comput., 14(Pt. B), pp. 289–304. [CrossRef]
Zhang, J. , Zhu, Q. , Wu, X. , and Li, Y. , 2013, “ A Generalized Indirect Adaptive Neural Networks Backstepping Control Procedure for a Class of Non-Affine Nonlinear Systems With Pure-Feedback Prototype,” Neurocomputing, 121(9), pp. 131–139.
Castañeda, C. , Loukianov, A. , Sanchez, E. , and Castillo-Toledo, B. , 2013, “ Real-Time Torque Control Using Discrete-Time Recurrent High-Order Neural Networks,” Neural Comput. Appl., 22(6), pp. 1223–1232. [CrossRef]
An, S. Q. , Lu, T. , and Ma, Y. , 2010, “ Simple Adaptive Control for SISO Nonlinear Systems Using Neural Network Based on Genetic Algorithm,” IEEE 9th International Conference on Machine Learning and Cybernetics, Qingdao, China, July 11–14, pp. 981–986.
Perez-Cruz, J. H. , Rubio, J. J. , Linares-Flores, J. , and Rangel, E. , 2015, “ Control of Uncertain Plants With Unknown Deadzone Via Differential Neural Networks,” IEEE Lat. Am. Trans., 13(7), pp. 2085–2093. [CrossRef]
Zhang, H. , Wu, W. , and Yao, M. , 2012, “ Boundedness and Convergence of Batch Back-Propagation Algorithm With Penalty for Feedforward Neural Networks,” Neurocomputing, 89, pp. 141–146. [CrossRef]
Loghmanian, S. , Jamaluddin, H. , Ahmad, R. , Yusof, R. , and Khalid, M. , 2012, “ Structure Optimization of Neural Network for Dynamic System Modeling Using Multi-Objective Genetic Algorithm,” Neural Comput. Appl., 21(6), pp. 1281–1295. [CrossRef]
Rubio, J. J. , 2014, “ Evolving Intelligent Algorithms for the Modelling of Brain and Eye Signals,” Appl. Soft Comput., 14(Pt. B), pp. 259–268. [CrossRef]
Xie, W. , Zhu, Y. , Zhao, Z. , and Wong, Y. , 2009, “ Nonlinear System Identification Using Optimized Dynamic Neural Network,” Neurocomputing, 72(13–15), pp. 3277–3287. [CrossRef]
Ge, H. , Qian, F. , Liang, Y. , Du, W. , and Wang, L. , 2008, “ Identification and Control of Nonlinear Systems by a Dissimilation Particle Swarm Optimization-Based Elman Neural Network,” Nonlinear Anal.: Real World Appl., 9(4), pp. 1345–1360. [CrossRef]
Coelho, L. , and Wicthoff, M. , 2009, “ Nonlinear Identification Using a B-Spline Neural Network and Chaotic Immune Approaches,” Mech. Syst. Signal Process., 23(8), pp. 2418–2434. [CrossRef]
Subudhi, B. , and Jenab, D. , 2011, “ A Differential Evolution Based Neural Network Approach to Nonlinear System Identification,” Appl. Soft Comput., 11(1), pp. 861–871. [CrossRef]
Romero-Ugalde, H. M. , Carmona, J.-C. , Alvarado, V. M. , and Reyes-Reyes, J. , 2013, “ Neural Network Design and Model Reduction Approach for Black Box Nonlinear System Identification With Reduced Number of Parameters,” Neurocomputing, 101, pp. 170–180. [CrossRef]
Romero-Ugalde, H. M. , Carmona, J.-C. , Alvarado, V. M. , Reyes-Reyes, J. , and Corbier, C. , 2014, “ Balanced Simplicity-Accuracy Neural Network Model Families for System Identification,” Neural Comput. Appl., 26(1), pp. 171–186. [CrossRef]
Romero-Ugalde, H. M. , 2013, “ System Identification Using Neural Networks: A Balanced Accuracy, Complexity and Computational Cost Approach,” Ph.D. thesis, in collaboration with the “Ecole nationale supérieure d'arts et métiers (ENSAM), France” and the “Centro Nacional de Investigacion y Desarrollo Tecnologico (CENIDET), Mexico”.
Romero-Ugalde, H. M. , Carmona, J.-C. , Reyes-Reyes, J. , Alvarado, V. M. , and Mantilla, J. , 2015, “ Computational Cost Improvement of Neural Network Models in Black Box Nonlinear System Identification,” Neurocomputing, 166, pp. 96–108. [CrossRef]
Xu, B. , Yang, C. , and Shi, Z. , 2014, “ Reinforcement Learning Output Feedback NN Control Using Deterministic Learning Technique,” IEEE Trans. Neural Networks Learn. Syst., 25(3), pp. 635–641. [CrossRef]
Xu, B. , Shi, Z. , Yang, C. , and Sun, F. , 2014, “ Composite Neural Dynamic Surface Control of a Class of Uncertain Nonlinear Systems in Strict-Feedback Form,” IEEE Trans. Cybern., 44(12), pp. 2626–2634. [CrossRef] [PubMed]
Sahnoun, M. A. , Romero-Ugalde, H. M. , Carmona, J.-C. , and Gomand, J. , 2013, “ Maximum Power Point Tracking Using P&O Control Optimized by a Neural Network Approach: A Good Compromise Between Accuracy and Complexity,” Energy Procedia, 42, pp. 650–659. [CrossRef]
Xu, B. , Yang, C. , and Pan, Y. , 2015, “ Global Neural Dynamic Surface Tracking Control of Strict-Feedback Systems With Application to Hypersonic Flight Vehicle,” IEEE Trans. Neural Networks Learn. Syst., 26(10), pp. 2563–2575. [CrossRef]


Grahic Jump Location
Fig. 1

Recurrent 2nn-2-1 neural network

Grahic Jump Location
Fig. 2

Recurrent 2-1 neural network

Grahic Jump Location
Fig. 3

Recurrent 2nn-1 neural network

Grahic Jump Location
Fig. 4

(a) Experimental piezoelectric system setup, (b) measurements chain, and (c) experimental setup characteristics

Grahic Jump Location
Fig. 5

Frequency response function (mixed L2/L1-norms versus L2-norm)

Grahic Jump Location
Fig. 6

Comparison with nonlinear models in terms of FRF

Grahic Jump Location
Fig. 7

Comparison with pseudolinear models in terms of FRF



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In