Group method of data handling

Lore & Background

The method was originated in 1968 by Prof. Alexey G. Ivakhnenko at the Institute of Cybernetics in Kyiv. From the very beginning, it was a computer-based method, with a set of computer programs and algorithms as primary practical results. Thanks to the author's policy of open code sharing, the method quickly settled in a large number of scientific laboratories worldwide. The development of GMDH consists of a synthesis of ideas from different areas of science: the cybernetic concept of 'black box' and the principle of successive genetic selection of pairwise features, Gödel's incompleteness theorems, Gabor's principle of 'freedom of decisions choice', and Beer's principle of external additions.

Reader's Guide

GMDH is significant as an original method for solving problems of structural-parametric identification of models for experimental data under uncertainty. It differs from other modelling methods by actively applying automatic models generation, inconclusive decisions, and consistent selection by external criteria for finding models of optimal complexity. Its multilayered procedure for automatic models structure generation imitates the process of biological selection with pairwise successive features, a procedure currently used in deep learning networks. The method established the foundations of noise-immune modelling, showing that the complexity of the optimal predictive model depends on the level of uncertainty in the data: the higher the noise, the simpler the optimal model. Jürgen Schmidhuber cites GMDH as one of the first deep learning methods, noting it was used to train eight-layer neural nets as early as 1971. This inductive approach minimized human influence on the objective result and can be considered an implementation of the Artificial Intelligence thesis that a computer can act as a powerful advisor to humans.

Did You Know?

• GMDH algorithms consider various component subsets of the base function called partial models, with coefficients estimated by the least squares method.
• The period 1968–1971 was characterized by the application of only the regularity criterion for solving problems of identification, pattern recognition, and short-term forecasting.

The Self-Organizing Engine: How GMDH Builds Models

GMDH operates as a family of inductive algorithms whose defining trait is self-organization. Rather than imposing a fixed architecture on the data, the method iteratively generates candidate models—typically polynomial in form—and evaluates each against an external performance criterion. The process begins with simple partial models, often limited to second-degree functions, and progressively adds components until the external criterion reaches its minimum, signalling that the model has achieved optimal complexity. Coefficients within each partial model are estimated via the least squares method, and the algorithm draws from a base function originally expressed as the Kolmogorov–Gabor polynomial, which expands through successive interaction terms of the input variables. Because the system continuously adapts to the noise present in empirical data, it naturally resists overfitting while preserving generalizability. The result is a feedforward network whose structure and parameters emerge entirely from the data itself, a property that has earned GMDH the alternative labels "heuristic self-organization of models" and "polynomial feedforward neural network."

Origins in Kyiv and the Convergence of Ideas

The method traces its birth to 1968, when Professor Alexey G. Ivakhnenko developed it at the Institute of Cybernetics in Kyiv. From its inception, GMDH was conceived as a computer-based procedure, meaning that the primary tangible outputs were not merely theoretical papers but working software and algorithmic toolkits. Ivakhnenko's deliberate policy of open code sharing accelerated the method's adoption across scientific laboratories around the world, allowing researchers to put the algorithms into practice without waiting for proprietary implementations. The intellectual architecture of GMDH is a remarkable synthesis: it weaves together the cybernetic notion of treating a system as a "black box," a principle of successive genetic selection applied to pairwise features, Gödel's incompleteness theorems, Gabor's principle of freedom in choosing decisions, and Beer's principle of external additions. This multi-disciplinary foundation gave the method a philosophical depth that set it apart from purely statistical modelling approaches of the era.

A Pioneer of Automated and Deep Learning

Although GMDH predates the modern deep learning revolution by decades, its influence on the field is well documented. Jürgen Schmidhuber, a prominent figure in neural network research, has explicitly cited GMDH as one of the earliest deep learning methods, noting that practitioners were already training eight-layer neural networks using the technique as early as 1971. This places GMDH firmly among the first approaches to what we now call automated machine learning, where the algorithm itself decides the architecture rather than a human engineer specifying every layer and connection. The method's inductive character—its capacity to uncover patterns and interdependencies in data without demanding strong a priori assumptions—anticipates the philosophy behind many contemporary learning systems. By balancing structural complexity against predictive accuracy through its self-organization mechanism, GMDH established a design principle that continues to resonate in modern model selection and regularization strategies.

The Machine as Advisor: Philosophy and Practical Reach

GMDH carries a philosophical commitment that distinguishes it from many modelling traditions: the computer is not merely a calculator but an active advisor to human decision-makers. Because the vast majority of routine analytical work—generating candidate structures, estimating coefficients, evaluating performance criteria—is delegated to the algorithm, the potential for subjective human bias to distort the final result is substantially reduced. This aligns with what the source material describes as an implementation of the broader Artificial Intelligence thesis, in which a machine serves as a powerful consultant rather than a passive tool. In practice, GMDH has found application across machine learning, forecasting, optimization, and pattern recognition, particularly in domains where relationships among variables are highly nonlinear and resist closed-form description. Its inductive, assumption-light approach makes it especially suited to complex systems where the analyst cannot specify in advance which interactions matter, allowing the data itself to reveal the most informative structure.