Compositional search
Building large models from smaller ones: surrogate-seeded starting points, automatic decomposition, and sub-searches that solve one part at a time.
Models are made of smaller models, and the search has to know that
A symbolic regression candidate that matches real-world data is rarely one monolithic expression. It is usually a composition: a handful of terms that each explain a part, a linear combination of a few nonlinear features, a piecewise structure. Treating the search as if every candidate were a single tree of arithmetic throws away structure that the search could be using as a prior.
Ufinq treats composition as a first-class concept. Where a first candidate comes from is a named choice the search makes among several composition methods, and a hard target can be split so that one part is searched on its own. The research question is what to compose, with what, when.
Four threads of work
Where candidates come from
- Tree builders. Top-down, bottom-up, bidirectional and hybrid construction, and PTC2: classical ways of drawing a tree from the function space under size constraints.
- Surrogate-seeded composition. A small random forest, a single-hidden-layer network or a kernel regression is trained on the data and translated into a symbolic expression. The search does not keep the surrogate; it keeps a readable fragment of what the surrogate found.
- Affine and skeleton starts. A scaled linear form over the inputs, and structural skeletons whose constants the optimizers fill in.
- A weighted mix. Every draw picks one of these methods by weight.
- Open questions. Learning the mix per dataset; when a surrogate seed helps and when it anchors the search on the wrong shape.
Automatic decomposition
- Signal-driven. No one has to declare the parts. The search notices which component of a candidate carries the error and treats that component as a problem of its own.
- Sub-search. A mutation opens an inner search on a localized dataset for that one component and splices the result back into the candidate.
- Structured targets. Vector, tuple and record targets are built from per-part expressions that are evolved jointly.
- Open questions. Decomposition driven by the coordinating branch across the cluster; keeping sub-results from collapsing diversity at the outer level.
Combination crossovers
- Function combination. Takes a node from each parent and joins them under a new function that accepts both.
- Model combination. The same, joined under a reference to a named sub-model.
- Family discipline. Both belong to the Combination crossover family; the adaptive operator bandit weighs them against the other operators from per-branch outcomes.
- Open questions. Combination that preserves the refinement state of the parts it joins; interaction with sub-searches within the same generation.
A library of sub-models (experimental)
- Modelbase. A store of sub-expressions harvested from the best candidates when a branch finishes, indexed by structure.
- Status. Harvesting and lookup exist; reuse across searches is not switched on. This is a research thread, not a property of the system today.
- Open questions. Ranking when several sub-models match; preventing runs that converge on a named sub-model in a thin wrapper.
Three consequences of this design
Related material
The system view of how composition fits into a single branch's lifecycle is described under Technology § 1. The function space the composition methods draw from is described under Technology § 4.
Adjacent areas: Genetic operators describes the broader operator taxonomy that the combination crossovers and the sub-search mutation fit into; Distributed orchestration describes how work is delegated across the cluster.
Continue reading
Other research areas are listed under Research. Published work lives on Papers; short-form notes on Notes.