This article presents a novel approach for constructing self-optimizing algorithms designed
to estimate parameters (hyperparameters) in complex systems, with a broader application to
classical genetic and evolutionary algorithms. The central theme of this paper revolves around
the exploration of multimodality in the objective function and advocates the effectiveness of
employing distribution mixtures as opposed to single-peaked distributions in traditional scenarios.
A significant focus of this research involves addressing the challenge of determining the
dimensionality of the mixture and developing algorithms for both augmenting and reducing it.
The methods employed for manipulating the mixture’s dimensionality are inspired by cluster
analysis techniques, specifically those utilized in the CURE and BIRCH big data clustering
algorithms. Furthermore, this work delves into a detailed examination of a self-adaptive algorithm
grounded in a mixture of distributions, illustrated by the CMA-ES algorithm. It is evident
that the proposed approach outlined in this paper exhibits versatility, making it applicable not
only to the CMA-ES algorithm but also to various optimization algorithms involved in tasks
such as classification or regression recovery.

[1] Sakamoto, N., & Akimoto, Y. (2017). Modified box constraint handling for the covariance matrix
adaptation evolution strategy. Proceedings of the Genetic and Evolutionary Computation Conference
Companion on - GECCO ’17. https://doi.org/10.1145/3067695.3075986
[2] Dang, V.-H., Vien, N. A., & Chung, T. (2019). A covariance matrix adaptation evolution
strategy in reproducing kernel Hilbert space. Genetic Programming and Evolvable Machines.
https://doi.org/10.1007/s10710-019-09357-1
[3] Roeva, O.; Zoteva, D.;Roeva, G.; Lyubenova, V. (2023). An Efficient Hybrid of an Ant Lion Optimizer
an dGenetic Algorithm for a Model Parameter Identification Problem.Mathematics, 11, 1292.
https://doi.org/10.3390/math11061292
[4] Albadr, M. A., Tiun, S., Ayob, M., & AL-Dhief, F. (2020). Genetic Algorithm Based on Natural Selection
Theory for Optimization Problems. Symmetry, 12(11), 1758. https://doi.org/10.3390/sym12111758
[5] Xuefeng, W., & Chen, M. (2021). Application of Mathematical Model Based on Optimization Theory
and Particle Swarm Algorithm in Radar Station Layout Optimization. Journal of Physics: Conference
Series, 1848(1), 012087. https://doi.org/10.1088/1742-6596/1848/1/012087
[6] Dorsey, Robert & Mayer, Walter. (1995). Genetic Algorithms for Estimation Problems With Multiple
Optima, Nondifferentiability, and Other Irregular Features. Journal of Business & Economic Statistics.
13. 53-66. https://doi.org/10.1080/07350015.1995.10524579.
[7] Alhijawi, Bushra & Awajan, Arafat. (2023). Genetic algorithms: theory, genetic operators, solutions,
and applications. Evolutionary Intelligence. 1-12. 10.1007/s12065-023-00822-6.
[8] Hansen, Nikolaus & M¨uller, Sibylle & Koumoutsakos, Petros. (2003). Reducing the Time Complexity
of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES). Evolutionary
computation. 11. 1-18. https://doi.org/10.1162/106365603321828970.
[9] Ilya Loshchilov, Frank Hutter. CMA-ES for Hyperparameter Optimization of Deep NeuralNetworks.
(2016). arXiv:1604.07269v1 [cs.NE] 25. – 9p. https://doi.org/10.48550/arXiv.1604.07269
[10] Hansen, Nikolaus & Ros, Raymond & Mauny, Nikolas & Schoenauer, Marc & Auger, Anne. (2011).
Impacts of Invariance in Search: When CMA-ES and PSO Face Ill-Conditioned and Non-Separable
Problems. Applied Soft Computing. 11. https://doi.org/10.1016/j.asoc.2011.03.001.
[11] Malyk I.V., Litvinchuk Yu.A. (2023). The extended CMA-ES algorithm. Bukovinian Math Journal. 10,
2, 137-143. https://doi.org/10.31861/bmj2022.02.09.
[12] Sundberg, Rolf (2019). Statistical Modelling by Exponential Families. Cambridge University Press.
[13] Lorbeer, Boris & Kosareva, Ana & Deva, Bersant & Softi´c, Dˇzenan & Ruppel, Peter &
K¨upper, Axel. (2017). Variations on the Clustering Algorithm BIRCH. Big Data Research. 11.
10.1016/j.bdr.2017.09.002.
[14] Lang, Andreas & Schubert, Erich. (2020). BETULA: Numerically Stable CF-Trees for BIRCH Clustering.
[15] Kogan, Jacob; Nicholas, Charles K.; Teboulle, M. (2006). Grouping multidimensional data: recent
advances in clustering. Springer.
[16] Guha, Sudipto & Rastogi, Rajeev & Shim, Kyuseok. (1998). CURE: An efficient clustering algorithm
for large databases. Information Systems. 26. 35-58. 10.1016/S0306-4379(1)00008-4.
[17] Qian, Yun-Tao & Shi, Qing-Song & Wang, Qi. (2002). CURE-NS: a hierarchical clustering algorithm
with new shrinking scheme. 2. 895 - 899 vol.2. 10.1109/ICMLC.2002.1174512.