Subject: Re: [boost] Proposal for a Differential Evolution C++ library
From: Adrian Michel (adrian_at_[hidden])
Date: 2012-01-10 16:33:51
On 1/9/2012 12:09 PM, Simonson, Lucanus J wrote:
>
>
> I think that numerical optimization is more general than its application to
> machine learning. While you argue that you'd like to see the library go
> deep in machine learning I don't see anything wrong with a general purpose
> numerical optimization library. However, DE is more general than numerical
> optimization, while the library seems to be an application of DE to
> numerical optimization. I'm not sure that DE is even one of the best ways
> to do numerical optimization (I'm pretty sure genetic algorithm is a bad
> way, so I'm skeptical), and I know I'd like a more general interface to a DE
> engine than one designed for numerical optimization problems. I'd like to
> see the library go wide for max applicability rather than deep. Hopefully
> that will keep it out of all the different cans of worms that could be
> opened in each application domain. Put numerical optimization and machine
> learning applications in the example code of the library and submit just the
> core engine as the library itself. Narrowing the scope of a library submission
> is usually the path to success.
>
I forwarded Ken Price (one of the DE co-authors) some of the comments I
received for his view on things and here's his reply, which hopefully
should clarify some of the issues that have been raised:
"The skeptic is right in believing that GAs have traditionally done
poorly at numerical optimization, primarily because they rely on general
data manipulation operations (bit flips, exchanges, rotations,
insertions, replacements...) that are not appropriate for
real-parameter optimization. DE, however, was specifically designed for
numerical optimization (although it has since been applied with limited
success to combinatorial problems). Instead of relying on general data
manipulation operations, DE's unique differential mutation operation is
a real-valued vector operation that is appropriate for the numerical
optimization problem domain. More than just appropriate, the
differential mutation operation at the core of DE is effective because
it exploits the tendency of a population to distribute itself around
function contours. Randomly sampling the difference in the location of
population members when they are distributed along a function's level
surfaces turns out to be a very effective way to deal with some of the
most challenging aspects of numerical optimization, e.g. multiple
optima, parameter-dependence, slow-down at high resolution and disparate
parameter sensitivities.
As evidence of DE's utility as a numerical optimizer, I would offer that
it has been part of Mathematica's numerical optimization toolbox for
over a decade. It helped the Mathematica team win the SIAM 100-digit
challenge (ten problems to ten-digit accuracy) by solving what the
contest participants agreed was the contest's most difficult (numerical
optimization) problem. (DE also solved one of the other problems in the
contest). In the many other contests that have been held to compare
numerical optimizers, DE has consistently placed at or near the top.
DE is also part of the OPTIMUS package, which is highly respected and in
wide use by several automobile manufactures. Alternatively, skeptics can
do an Internet search and discover for themselves the enormous range of
engineering problems the DE has successfully solved, nearly all of which
are numerical optimization problems, often with multiple, non-linear
constraints. DE has also proven effective in the domain of
multi-objective numerical optimization.
Finally, I would suggest that since DE is so simple and easy to
implement, skeptics ought to just try it. Most people are surprised by
how effective something as simple as DE can be."