This is actually similar to a common approach in probabilistic modeling.
Pick an initial model & set of probabilistic priors. Evaluate it with a "goodness of fit" heuristic function, then iterate & keep measuring while keeping the best solution discovered.
As long as the initial parameters are sort of reasonable, it will give you pretty good results for many problems.
It obviously doesn't get rid of the need for a better understanding of the problem. Improvements to how well your features describe important attributes of the problem tend to be strictly superior to your choice of learning algorithm (i.e. your exact process of iteration).
But as long as the initial solution is more or less on target? You can solve many problems by picking a solution that you know is inadequate, then iterating.
Pick an initial model & set of probabilistic priors. Evaluate it with a "goodness of fit" heuristic function, then iterate & keep measuring while keeping the best solution discovered.
As long as the initial parameters are sort of reasonable, it will give you pretty good results for many problems.
It obviously doesn't get rid of the need for a better understanding of the problem. Improvements to how well your features describe important attributes of the problem tend to be strictly superior to your choice of learning algorithm (i.e. your exact process of iteration).
But as long as the initial solution is more or less on target? You can solve many problems by picking a solution that you know is inadequate, then iterating.