[Ifeffit] Chi in arthemis

[Matt Newville]

Hi Scott,

> Is this the most recent IXAS report on error reporting standards?
>
> http://www.i-x-s.org/OLD/subcommittee_reports/sc/err-rep.pdf

Yes.  To be clear, the main value of reduced chi-square is that it can
be used (even if with some inherent uncertainty) to compare two models
with different number of variables.  Many analysis programs report
only a value like R-factor (ie, the misfit not scaled by the
measurement uncertainty or number of free parameters in the data).
Again, this is an OK measure of the misfit, though it too is scaled
somewhat arbitrarily, and cannot be used to compare models with
different number of variables.

> ... In such cases I have become convinced that the R-factor alone
> provides as much meaningful information as the chi-square values,
> and that in fact the chi-square values can be confusing when listed
> for fits on different data. For those working with dilute samples,
> on the other hand, I can see that chi-square might be a meaningful
> quantity.
>
> ... I strongly agree that the decision of which measurements
> of quality of fit to produce should not be dependent on what "looks
> good"! That would be bad science. The decision of what figures of
> merit to present should be made a priori.

The subcommittee that looked into agreed (after some debate) on
wording and recommendations of such topics also thought it should be
done a priori, though they also thought is should be done without
regard to quality of the data or type of samples.   You're free to
disagree with this report.

--Matt