12.3. minimize() and objective Functions

As mentioned above, the objective function returns an array calculated from given a group of parameters. This array will be minimized in the least-squares sense in the fitting process. For most fits, the objective function should return the residual array (data - model), given a group of parameters and optional inputs. You’ll note that we didn’t explicitly mention any data in describing the objective function. This is because, formally, the minimization process may be looking for a solution to a purely mathematical problem, not just fitting to data. Even when the objective function does return the difference of data and model, the data to be modeled may be quite complex. It might, for example, be contained in two or more arrays – perhaps what you want to model is the difference of two image arrays, or the fourier filtered average of ten spectra. Because of such complexities, the reliance of optional arguments appears to be the best approach.

A simple objective function that models data as a line might look like this:

params = param_group(offset = param(0., vary=True),
                     slope = param(200, min=0, vary=True))

def residual(pars, xdata=None, ydata=None):
    model = pars.offset + pars.slope * xdata
    diff  = ydata - model
    return diff
enddef

Here params is a Group containing two Parameters as defined by _math.param(), discussed earlier.

To actually perform the fit, the minimize() function must be called. This takes the objective function as its first argument, and the group containing all the Parameters as its second argument, keyword arguments for the optional arguments for the objective function, and several keyword arguments to alter the fitting process itself. Here is the function call using the objective function defined above, assuming you have a group called data containing the data you are trying to fit:

result = minimize(residual, params, kws={'xdata': data.x, 'ydata':data.y})

As the fit proceeds, the values the Parameter values will be updated, and the objective function will be called to recalculate the residual array. Thus the objective function may be called many times before the fitting procedure decides it has found the best solution that it can.

Changed in version 0.9.34: minimize() returns a result group containing fit statistics.

minimize(fcn, paramgroup, args=None, kws=None, method='leastsq', **extra_kws)

find the best-fit values for the Parameters in paramgroup such that the output array from the objective function fcn() has minimal sum-of-squares.

Parameters:

• fcn – objective function, which must have signature and output as described below.

• paramgroup – a Group containing the Parameters used by the objective function. This will be passed as the first argument to the objective function. The Group can contain other components in addition to the set of Parameters for the model.

• args – a tuple of positional arguments to pass to the objective function. Note that a single argument tuple is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses)

• kws – a dictionary of keyword/value arguments to pass to the objective function.

• method – name (case insensitive) of minimization method to use (default=’leastsq’)

• extra_kws – additional keywords to pass to fitting method.

Returns:

a Group containing several fitting statisics and best-fit parameters.

The method argument gives the name of the fitting method to be used. Several methods are available, as described below in Table of Fitting Methods.

Table of Fitting Methods.

Listed are the names and description of fitting methods available to the minimize() function. The leastsq method is the default, and the only method for which uncertainties and correlations are automatically calculated.

method name	Description
Leastsq	Levenberg-Marquardt.
Nelder-Mead	Nelder-Mead downhill simplex.
Powell	Powell’s method.
BFGS	quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno.
CG	Conjugate Gradient.
LBFGSB	Limited-Memory BFGS Method with Constraints.
TNC	Truncated Newton method.
COBYLA	Constrained Optimization BY Linear Approximation.
SLSQP	Sequential Least SQuares Programming.

Further information on these methods, including full lists of extra parameters that can be passed to them, can be found in the lmfit documentation.

It should be noted that the Levenberg-Marquardt algorithm is almost always the fastest of the methods listed (often by 10x), and is generally fairly robust. It is sometimes criticized as being sensitive to initial guesses and prone to finding local minima. The other fitting methods use very different algorithms, and so can be used to explore these effects. Many of them are much slower – using more than ten times as many evaluations of the objective function is not unusual. This does not guarantee a more robust answer, but it does allow one to try out and compare the results of the different methods.

While the TNC, COBYLA, SLSQP, and LBFGSB methods are supported, their principle justification is that the underlying algorithms support constraints. For Larch, this advantage is not particularly important, as all fitting methods can have constraints applied through Parameters, and the mechanism used by the native methods is not actually even supported with Larch. That said, all these methods are still interesting to explore.

Extra keywords for the leastsq method include:

extra_kw arg for method='leastsq'	Default Value	Description
xtol	1.e-7	Relative error in the approximate solution
ftol	1.e-7	Relative error in the desired sum of squares
maxfev	2000*(nvar+1)	maximum number of function calls (nvar= # of variables)
Dfun	None	function to call for Jacobian calculation

By default, numerical derivatives are used, and the following arguments are used.