Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion
| datacite.rights | http://purl.org/coar/access_right/c_abf2 | |
| dc.contributor.author | Gelvez-Almeida, Elkin | |
| dc.contributor.author | Barrientos, Ricardo | |
| dc.contributor.author | Vilches, Karina | |
| dc.contributor.author | Mora, Marco | |
| dc.date.accessioned | 2025-02-05T20:17:06Z | |
| dc.date.available | 2025-02-05T20:17:06Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | Randomization-based neural networks have gained wide acceptance in the scientific community owing to the simplicity of their algorithm and generalization capabilities. Random vector functional link (RVFL) networks and their variants are a class of randomization-based neural networks. RVFL networks have shown promising results in classification, regression, and clustering problems. For real-world applications, learning algorithms that can train with new samples over previous results are necessary because of to the constant generation of problems related to large-scale datasets. Various online sequential algorithms, commonly involving an initial learning phase followed by a sequential learning phase, have been proposed to address this issue. This paper presents a training algorithm based on multiple online sequential random vector functional link (OS-RVFL) networks for large-scale databases using a shared memory architecture. The training dataset is distributed among p OS-RVFL networks, which are trained in parallel using p threads. Subsequently, the test dataset samples are classified using each trained OS-RVFL network. Finally, a frequency criterion is applied to the results obtained from each OS-RVFL network to determine the final classification. Additionally, an equation was derived to reasonably predict the total training time of the proposed algorithm based on the learning time in the initial phase and the time scaling factor compared to the sequential learning phase. The results demonstrate a drastic reduction in training time because of data distribution and an improvement in accuracy because of the adoption of the frequency criterion. | eng |
| dc.format.mimetype | ||
| dc.identifier.citation | Gelvez-Almeida, E., Barrientos, R.J., Vilches-Ponce, K. et al. Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion. Sci Rep 14, 16104 (2024). https://doi.org/10.1038/s41598-024-66676-9 | eng |
| dc.identifier.doi | https://doi.org/10.1038/s41598-024-66676-9 | |
| dc.identifier.issn | 20452322 | |
| dc.identifier.uri | https://hdl.handle.net/20.500.12442/16220 | |
| dc.language.iso | eng | |
| dc.publisher | Nature | spa |
| dc.rights | Attribution-NonCommercial-NoDerivs 3.0 United States | eng |
| dc.rights.accessrights | info:eu-repo/semantics/openAccess | |
| dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/3.0/us/ | |
| dc.source | Scientific reports | eng |
| dc.source | No. 14, 16104 (2024) | spa |
| dc.subject.keywords | Randomization-based neural | eng |
| dc.subject.keywords | Random vector functional link (RVFL) | eng |
| dc.subject.keywords | Neural Networks | eng |
| dc.subject.keywords | Training algorithm | eng |
| dc.subject.keywords | Sequential Learning | eng |
| dc.title | Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion | eng |
| dc.type.driver | info:eu-repo/semantics/article | |
| dc.type.spa | Artículo científico | spa |
| dcterms.references | Zhang, L. & Suganthan, P. N. A survey of randomized algorithms for training neural networks. Inf. Sci. 364–365, 146–155. https://doi.org/10.1016/j.ins.2016.01.039 (2016). | spa |
| dcterms.references | Suganthan, P. N. & Katuwal, R. On the origins of randomization-based feedforward neural networks. Appl. Soft Comput. 105, 107239. https://doi.org/10.1016/j.asoc.2021.107239 (2021). | spa |
| dcterms.references | Pao, Y.-H. & Takefuji, Y. Functional-link net computing: Theory, system architecture, and functionalities. Computer 25, 76–79. https://doi.org/10.1109/2.144401 (1992). | spa |
| dcterms.references | Pao, Y.-H., Park, G.-H. & Sobajic, D. J. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 163–180. https://doi.org/10.1016/0925-2312(94)90053-1 (1994). | spa |
| dcterms.references | Zhang, L. & Suganthan, P. N. A comprehensive evaluation of random vector functional link networks. Inf. Sci. 367, 1094–1105. https://doi.org/10.1016/j.ins.2015.09.025 (2016). | spa |
| dcterms.references | Gelvez-Almeida, E. et al. Parallel methods for linear systems solution in extreme learning machines: An overview. J. Phys. Conf. Ser. 1702, 012017. https://doi.org/10.1088/1742-6596/1702/1/012017 (2020). | spa |
| dcterms.references | Malik, A. K., Gao, R., Ganaie, M., Tanveer, M. & Suganthan, P. N. Random vector functional link network: Recent developments, applications, and future directions. Appl. Soft Comput. 143, 110377. https://doi.org/10.1016/j.asoc.2023.110377 (2022). | spa |
| dcterms.references | Liang, N.-Y., Huang, G.-B., Saratchandran, P. & Sundararajan, N. A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Netw. 17, 1411–1423. https://doi.org/10.1109/TNN.2006.880583 (2006). | spa |
| dcterms.references | Matias, T., Souza, F., Araújo, R., Gonçalves, N. & Barreto, J. P. On-line sequential extreme learning machine based on recursive partial least squares. J. Process Control 27, 15–21. https://doi.org/10.1016/j.jprocont.2015.01.004 (2015). | spa |
| dcterms.references | Mirza, B., Lin, Z., Cao, J. & Lai, X. Voting based weighted online sequential extreme learning machine for imbalance multi-class classification. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS) 565–568. https://doi.org/10.1109/ISCAS.2015.7168696 (IEEE, 2015). | spa |
| dcterms.references | Gelvez-Almeida, E., Barrientos, R. J., Vilches-Ponce, K. & Mora, M. Parallel training of a set of online sequential extreme learning machines. In: 41st International Conference of the Chilean Computer Science Society (SCCC), Santiago. Chile 1–4, 2022. https://doi.org/10.1109/SCCC57464.2022.10000361 (21-25 November 2022). | spa |
| dcterms.references | Gelvez-Almeida, E., Barrientos, R. J., Vilches-Ponce, K. & Mora, M. Parallel model of online sequential extreme learning machines for classification problems with large-scale databases. In XI Jornadas de Cloud Computing, Big Data & Emerging Topics, La Plata, Argentina 19–23. http://sedici.unlp.edu.ar/handle/10915/155423 (27–29 June 2023). | spa |
| dcterms.references | Wibawa, I. P. D., Machbub, C., Rohman, A. S. & Hidayat, E. Modified online sequential extreme learning machine algorithm using model predictive control approach. Intell. Syst. Appl. 18, 200191. https://doi.org/10.1016/j.iswa.2023.200191 (2023). | spa |
| dcterms.references | Chen, Y. & Li, M. An effective online sequential stochastic configuration algorithm for neural networks. Sustainability 14, 15601. https://doi.org/10.3390/su142315601 (2022). | spa |
| dcterms.references | Zhang, X., Ma, H., Zuo, W. & Luo, M. Adaptive control of discrete-time nonlinear systems using ITF-ORVFL. IEEE/CAA J. Autom. Sin. 9, 556–563. https://doi.org/10.1109/JAS.2019.1911801 (2022). | spa |
| dcterms.references | Zha, L., Ma, K., Li, G., Yang, J. & Fang, Q. An improved extreme learning machine with self-recurrent hidden layer. Adv. Eng. Inform. 54, 101736. https://doi.org/10.1016/j.aei.2022.101736 (2022). | spa |
| dcterms.references | Kale, A. P., Sonawane, S., Wahul, R. M. & Dudhedia, M. A. Improved genetic optimized feature selection for online sequential extreme learning machine. Ingénierie des Systèmes d’Informationhttps://doi.org/10.18280/isi.270519 (2022). | spa |
| dcterms.references | Polat, Ö. & Kayhan, S. K. GPU-accelerated and mixed norm regularized online extreme learning machine. Concurr. Comput. Pract. Exp. 34, e6967. https://doi.org/10.1002/cpe.6967 (2022). | spa |
| dcterms.references | Ren, Y., Zhang, L. & Suganthan, P. N. Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput. Intell. Mag. 11, 41–53. https://doi.org/10.1109/MCI.2015.2471235 (2016). | spa |
| dcterms.references | Lan, Y., Soh, Y. C. & Huang, G.-B. Ensemble of online sequential extreme learning machine. Neurocomputing 72, 3391–3395. https://doi.org/10.1016/j.neucom.2009.02.013 (2009). | spa |
| dcterms.references | Liu, N. & Wang, H. Ensemble based extreme learning machine. IEEE Signal Process. Lett. 17, 754–757. https://doi.org/10.1109/LSP.2010.2053356 (2010). | spa |
| dcterms.references | Liu, Y. et al. Particle swarm optimization based selective ensemble of online sequential extreme learning machine. Math. Problems Eng.https://doi.org/10.1155/2015/504120 (2015). | spa |
| dcterms.references | Wei, L., Wang, L., Li, Y. & Duan, S. Ensemble of online sequential extreme learning machine based on cross-validation. J. Phys. Conf. Ser. 1550, 032156. https://doi.org/10.1088/1742-6596/1550/3/032156 (2020). | spa |
| dcterms.references | Zhai, J., Wang, J. & Wang, X. Ensemble online sequential extreme learning machine for large data set classification. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC) 2250–2255. https://doi.org/10.1109/SMC.2014.6974260 (IEEE, 2014). | spa |
| dcterms.references | Alhamdoosh, M. & Wang, D. Fast decorrelated neural network ensembles with random weights. Inf. Sci. 264, 104–117. https://doi.org/10.1016/j.ins.2013.12.016 (2014). | spa |
| dcterms.references | Mirza, B., Lin, Z. & Liu, N. Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149, 316–329. https://doi.org/10.1016/j.neucom.2014.03.075 (2015). | spa |
| dcterms.references | Ling, Q.-H., Song, Y.-Q., Han, F., Yang, D. & Huang, D.-S. An improved ensemble of random vector functional link networks based on particle swarm optimization with double optimization strategy. Plos ONE 11, e0165803. https://doi.org/10.1371/journal.pone.0165803 (2016). | spa |
| dcterms.references | Huang, S. et al. Parallel ensemble of online sequential extreme learning machine based on MapReduce. Neurocomputing 174, 352–367. https://doi.org/10.1016/j.neucom.2015.04.105 (2016). | spa |
| dcterms.references | Rakesh, K. & Suganthan, P. N. An ensemble of kernel ridge regression for multi-class classification. Procedia Comput. Sci. 108, 375–383. https://doi.org/10.1016/j.procs.2017.05.109 (2017). | spa |
| dcterms.references | Zhang, L. & Suganthan, P. N. Benchmarking ensemble classifiers with novel co-trained kernel ridge regression and random vector functional link ensembles [research frontier]. IEEE Comput. Intell. Mag. 12, 61–72 (2017). | spa |
| dcterms.references | Li, Y., Zhang, S., Yin, Y., Xiao, W. & Zhang, J. Parallel one-class extreme learning machine for imbalance learning based on Bayesian approach. J. Ambient Intell. Humaniz. Comput.https://doi.org/10.1007/s12652-018-0994-x (2018). | spa |
| dcterms.references | Katuwal, R. & Suganthan, P. N. Dropout and dropconnect based ensemble of random vector functional link neural network. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI) 1772–1778. https://doi.org/10.1109/SSCI.2018.8628640 (2018). | spa |
| dcterms.references | Liu, Y. et al. Ensemble neural networks with random weights for classification problems. In 2020 3rd International Conference on Algorithms, Computing and Artificial Intelligence 1–5. https://doi.org/10.1145/3446132.3446147 (2020). | spa |
| dcterms.references | Hu, M., Shi, Q., Suganthan, P. N. & Tanveer, M. Adaptive ensemble variants of random vector functional link networks. In Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 18–22, 2020, Proceedings, Part V 27 30–37. https://doi.org/10.1007/978-3-030-63823-8_4 (2020). | spa |
| dcterms.references | Malik, A. K., Ganaie, M., Tanveer, M. & Suganthan, P. N. A novel ensemble method of RVFL for classification problem. In 2021 International Joint Conference on Neural Networks (IJCNN) 1–8. https://doi.org/10.1109/IJCNN52387.2021.9533836 (2021). | spa |
| dcterms.references | Tanveer, M., Ganaie, M. & Suganthan, P. N. Ensemble of classification models with weighted functional link network. Appl. Soft Comput. 107, 107322. https://doi.org/10.1016/j.asoc.2021.107322 (2021). | spa |
| dcterms.references | Shi, Q., Katuwal, R., Suganthan, P. N. & Tanveer, M. Random vector functional link neural network based ensemble deep learning. Pattern Recognit. 117, 107978. https://doi.org/10.1016/j.patcog.2021.107978 (2021). | spa |
| dcterms.references | Dong, Q., Zhou, Y., Lian, J. & Li, L. Online adaptive humidity monitoring method for proton exchange membrane fuel cell based on fuzzy c-means clustering and online sequence extreme learning machine. Electrochim. Acta 429, 141059. https://doi.org/10.1016/j.electacta.2022.141059 (2022). | spa |
| dcterms.references | Yao, Y., Ding, J., Zhao, C., Wang, Y. & Chai, T. Data-driven constrained reinforcement learning for optimal control of a multistage evaporation process. Control Eng. Pract. 129, 105345. https://doi.org/10.1016/j.conengprac.2022.105345 (2022). | spa |
| dcterms.references | Li, R., Gao, R., Cui, J., Suganthan, P. & Sourina, O. Advanced ensemble deep random vector functional link for eye-tracking-based situation awareness recognition. In 2022 IEEE Symposium Series on Computational Intelligence (SSCI) 300–307. https://doi.org/10.1109/SSCI51031.2022.10022019 (2022). | spa |
| dcterms.references | Ganaie, M. & Tanveer, M. Ensemble deep random vector functional link network using privileged information for Alzheimer’s disease diagnosis. IEEE/ACM Trans. Comput. Biol. Bioinform.https://doi.org/10.1109/TCBB.2022.3170351 (2022). | spa |
| dcterms.references | Gao, R., Du, L., Suganthan, P. N., Zhou, Q. & Yuen, K. F. Random vector functional link neural network based ensemble deep learning for short-term load forecasting. Expert Syst. Appl. 206, 117784. https://doi.org/10.1016/j.eswa.2022.117784 (2022). | spa |
| dcterms.references | Yu, X. et al. Selective ensemble deep bidirectional RVFLN for landslide displacement prediction. Nat. Hazards 112, 725–745. https://doi.org/10.1007/s11069-021-05202-w (2022). | spa |
| dcterms.references | Jamei, M. et al. Development of wavelet-based kalman online sequential extreme learning machine optimized with boruta-random forest for drought index forecasting. Eng. Appl. Artif. Intell. 117, 105545. https://doi.org/10.1016/j.engappai.2022.105545 (2023). | spa |
| dcterms.references | Zhou, X., Huang, J., Lu, F., Zhou, W. & Liu, P. A novel compound fault-tolerant method based on online sequential extreme learning machine with cycle reservoir for turbofan engine direct thrust control. Aerosp. Sci. Technol. 132, 108059. https://doi.org/10.1016/j.ast.2022.108059 (2023). | spa |
| dcterms.references | Rajpal, A., Sehra, K., Mishra, A. & Chetty, G. A low-resolution real-time face recognition using extreme learning machine and its variants. Imaging Sci. J.https://doi.org/10.1080/13682199.2023.2183544 (2023). | spa |
| dcterms.references | Xu, Q., Wei, X., Bai, R., Li, S. & Meng, Z. Integration of deep adaptation transfer learning and online sequential extreme learning machine for cross-person and cross-position activity recognition. Expert Syst. Appl. 212, 118807. https://doi.org/10.1016/j.eswa.2022.118807 (2023). | spa |
| dcterms.references | Wang, F., Song, Y., Liu, C., He, A. & Qiang, Y. Multi-objective optimal scheduling of laminar cooling water supply system for hot rolling mills driven by digital twin for energy-saving. J. Process Control 122, 134–146. https://doi.org/10.1016/j.jprocont.2023.01.004 (2023). | spa |
| dcterms.references | Yu, L., Bai, Y. & Li, K. Lane-changing control of vehicle platoon based on OS-ELM environmental parameter identifier. IEEE Trans. Veh. Technol.https://doi.org/10.1109/TVT.2022.3214935 (2023). | spa |
| dcterms.references | Kouzehkalani Sales, A., Gul, E. & Safari, M. J. S. Online sequential, outlier robust, and parallel layer perceptron extreme learning machine models for sediment transport in sewer pipes. Environ. Sci. Pollut. Res. 30, 39637–39652. https://doi.org/10.1007/s11356-022-24989-0 (2023). | spa |
| dcterms.references | Samal, S. & Dash, R. Developing a novel stock index trend predictor model by integrating multiple criteria decision-making with an optimized online sequential extreme learning machine. Granul. Comput. 8, 411–440. https://doi.org/10.1007/s41066-022-00338-x (2023). | spa |
| dcterms.references | Duan, W. et al. Battery SOH estimation and RUL prediction framework based on variable forgetting factor online sequential extreme learning machine and particle filter. J. Energy Storage 65, 107322. https://doi.org/10.1016/j.est.2023.107322 (2023). | spa |
| dcterms.references | Shi, Q., Hu, M., Suganthan, P. N. & Katuwal, R. Weighting and pruning based ensemble deep random vector functional link network for tabular data classification. Pattern Recognit. 132, 108879. https://doi.org/10.1016/j.patcog.2022.108879 (2022). | spa |
| dcterms.references | Li, R. et al. A spectral-ensemble deep random vector functional link network for passive brain-computer interface. Expert Syst. Appl. 227, 120279. https://doi.org/10.1016/j.eswa.2023.120279 (2023). | spa |
| dcterms.references | Li, R. et al. An enhanced ensemble deep random vector functional link network for driver fatigue recognition. Eng. Appl. Artif. Intell. 123, 106237. https://doi.org/10.1016/j.engappai.2023.106237 (2023). | spa |
| dcterms.references | Xiao, S., Zhang, F. & Huang, X. Online thickness prediction of hot-rolled strip based on ISSA-OSELM. Int. J. Interact. Design Manuf. (IJIDeM) 16, 1089–1098. https://doi.org/10.1007/s12008-021-00833-6 (2022). | spa |
| dcterms.references | Li, Y., Zhang, J., Zhang, S. & Xiao, W. Dual ensemble online modeling for dynamic estimation of hot metal silicon content in blast furnace system. ISA Trans. 128, 686–697. https://doi.org/10.1016/j.isatra.2021.09.018 (2022). | spa |
| dcterms.references | Ren, Q., Li, M., Kong, T. & Ma, J. Multi-sensor real-time monitoring of dam behavior using self-adaptive online sequential learning. Autom. Constr. 140, 104365. https://doi.org/10.1016/j.autcon.2022.104365 (2022). | spa |
| dcterms.references | Zhou, Z., Ji, H. & Zhu, Z. Online sequential fuzzy dropout extreme learning machine compensate for sliding-mode control system errors of uncertain robot manipulator. Int. J. Mach. Learn. Cybern. 13, 2171–2187. https://doi.org/10.1007/s13042-022-01513-x (2022). | spa |
| dcterms.references | Ali, M. et al. Coupled online sequential extreme learning machine model with ant colony optimization algorithm for wheat yield prediction. Sci. Rep. 12, 5488. https://doi.org/10.1038/s41598-022-09482-5 (2022). | spa |
| dcterms.references | He, J. et al. Ensemble deep random vector functional link for self-supervised direction-of-arrival estimation. Eng. Appl. Artif. Intell. 120, 105831. https://doi.org/10.1016/j.engappai.2023.105831 (2023). | spa |
| dcterms.references | Gao, R., Li, R., Hu, M., Suganthan, P. N. & Yuen, K. F. Significant wave height forecasting using hybrid ensemble deep randomized networks with neurons pruning. Eng. Appl. Artif. Intell. 117, 105535. https://doi.org/10.1016/j.engappai.2022.105535 (2023). | spa |
| dcterms.references | Huang, G.-B., Zhu, Q.-Y. & Siew, C.-K. Extreme learning machine: A new learning scheme of feedforward neural networks. In 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), Budapest, Hungary, vol 2, 985–990. https://doi.org/10.1109/IJCNN.2004.1380068 (25–29 July 2004). | spa |
| dcterms.references | Safaei, A., Wu, Q. M. J., Akilan, T. & Yang, Y. System-on-a-chip (SoC)-based hardware acceleration for an online sequential extreme learning machine (OS-ELM). IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 38, 2127–2138. https://doi.org/10.1109/TCAD.2018.2878162 (2018). | spa |
| dcterms.references | Rosato, A., Altilio, R. & Panella, M. On-line learning of RVFL neural networks on finite precision hardware. In 2018 IEEE International Symposium on Circuits and Systems (ISCAS) 1–5. https://doi.org/10.1109/ISCAS.2018.8351399 (IEEE, 2018). | spa |
| dcterms.references | Decherchi, S., Gastaldo, P., Leoncini, A. & Zunino, R. Efficient digital implementation of extreme learning machines for classification. IEEE Trans. Circuits Syst. II: Express Briefs 59, 496–500. https://doi.org/10.1109/TCSII.2012.2204112 (2012). | spa |
| dcterms.references | Xia, M., Wang, J., Liu, J., Weng, L. & Xu, Y. Density-based semi-supervised online sequential extreme learning machine. Neural Comput. Appl. 32, 7747–7758. https://doi.org/10.1007/s00521-019-04066-3 (2020). | spa |
| dcterms.references | Hong, J.-H., Min, J.-K., Cho, U.-K. & Cho, S.-B. Fingerprint classification using one-vs-all support vector machines dynamically ordered with naï ve bayes classifiers. Pattern Recognition 41, 662–671. https://doi.org/10.1016/j.patcog.2007.07.004 (2008). | spa |
| dcterms.references | Henry, E. R. Classification and uses of finger prints (His Magjesty’s Stationery Office, London, 1922). | spa |
| dcterms.references | Zabala-Blanco, D., Mora, M., Barrientos, R. J., Hernández-García, R. & Naranjo-Torres, J. Fingerprint classification through standard and weighted extreme learning machines. Appl. Sci. 10, 4125. https://doi.org/10.3390/app10124125 (2020). | spa |
| dcterms.references | Zabala-Blanco, D., Mora, M., Hernández-García, R. & Barrientos, R. J. The extreme learning machine algorithm for classifying fingerprints. In 2020 39th International Conference of the Chilean Computer Science Society (SCCC) 1–8. https://doi.org/10.1109/SCCC51225.2020.9281232 (IEEE, 2020). | spa |
| dcterms.references | Deng, L. The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process. Mag. 29, 141–142. https://doi.org/10.1109/MSP.2012.2211477 (2012). | spa |
| dcterms.references | Dua, D. & Graff, C. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml (2017). | spa |