Search strategies
Navigating the universal function space: branch roles, an adaptive search space, and the two diversity checks that hold populations open.
Navigating a universal function space
Symbolic regression searches an open-ended space. There is no fixed parameter vector, no bounded architecture, no closed set of candidates to enumerate. Every function the language can express is in scope, and the language is universal. Selection pressure alone is too coarse a tool for that space: a sharp pressure collapses the population onto the first plausible local optimum; a soft pressure drifts forever.
Ufinq treats the shape of the search as an explicit concern. Branches take different roles, so the population is repeatedly pushed off whatever attractor it has settled into. The space each generation draws from adapts to the data and to what the population already contains. And the gene pool is held open by two diversity checks that look at different properties, structural form and behavior, so neither can be satisfied by the other.
Four threads of work
Branch roles
- Three roles. Fresh exploration; a successor recombined from a matched pair; a restart seeded from the Hall of Fame. All three optimize the same quality.
- One objective. An earlier design gave successors alternating strategies and separate gene pools. Measured against a single objective with roles, it showed no advantage and was removed.
- Open questions. How the mix of roles should shift as a run matures.
An adaptive search space
- Rebuilt every generation. The search space (which functions, which range of complexity, which error bound) is recomputed each generation from the data and the current population.
- Complexity follows the population. The range of sizes the search may sample moves with what the population has achieved.
- Data-driven function gating. Functions the data cannot support are left out.
- Open questions. Learning from failed shapes, so that a structure which produced a non-finite result becomes harder to sample again.
Two diversity checks
- Structural diversity. Candidates are compared on structure with constants ignored, so the same skeleton with different constants counts once. Refined populations cannot hide duplicates behind constant variation.
- Behavioral diversity. Measured on the candidate's predictions on validation data. Two candidates with different structure may behave identically; the behavioral check catches the case the structural one misses.
- Checked early and late. The structural check runs when a candidate is created and again after refinement, so a duplicate is caught before it costs a full refinement, and again if refinement turns it into one.
- Open questions. The choice of behavioral-similarity metric under different output distributions; how strict each check should be as the population matures.
Selecting the model
- A Hall of Fame, ranked by quality. The coordinating branch keeps the best candidates from every branch, ordered by a single quality value that prices size against accuracy.
- A final check. The top of the Hall of Fame is polished once more and screened. On small samples, ties go to the shorter formula; a candidate that misbehaves inside the range of the data (a pole, a non-finite value) is passed over.
- One model out. The first candidate that survives is the model. Nothing else is returned.
- Open questions. How much of the Hall of Fame the final check should examine.
Three consequences of this design
Related material
A short-form note on the two diversity dimensions lives at Diversity is two things, not one. The full lifecycle of a single branch is described under Technology § 1 with Figure 1; how the single returned model is chosen is summarized under Research § 3.
Adjacent areas: Genetic operators describes the operators a branch uses to generate offspring; Symbolic refinement describes the normalization that makes the structural check meaningful; Distributed orchestration describes how branches are paired and delegated across the cluster.
Continue reading
Other research areas are listed under Research. Published work lives on Papers; short-form notes on Notes.