[Ifeffit] Chi in arthemis
Scott Calvin
SCalvin at slc.edu
Tue Aug 18 22:30:22 CDT 2009
Matt,
Is this the most recent IXAS report on error reporting standards?
http://www.i-x-s.org/OLD/subcommittee_reports/sc/err-rep.pdf
It uses a rather expansive definition of epsilon, which explicitly
includes "imperfect" ab initio standards such as FEFF calculations. It
indicates that statistical methods such as that used by ifeffit for
estimating measurement error yields a lower limit for epsilon, and
thus an overestimate of chi square.
So I think my statement and yours are entirely compatible.
As far as what should be reported, I do deviate from the IXAS
recommendations by not reporting chi-square. Of course, I tend to work
in circumstances where the signal-to-noise ratio is very high, and
thus the statistical uncertainties make a very small contribution to
the overall measurement error. In such cases I have become convinced
that the R-factor alone provides as much meaningful information as the
chi-square values, and that in fact the chi-square values can be
confusing when listed for fits on different data. For those working
with dilute samples, on the other hand, I can see that chi-square
might be a meaningful quantity.
At any rate, I strongly agree that the decision of which measurements
of quality of fit to produce should not be dependent on what "looks
good"! That would be bad science. The decision of what figures of
merit to present should be made a priori.
--Scott Calvin
Sarah Lawrence College
On Aug 18, 2009, at 10:40 PM, Matt Newville wrote:
> Having a "reasonable R-factor" of a few percent misfit and a reduced
> chi-square of ~100 means the misfit is much larger than the estimated
> uncertainty in the data. This is not at all unusual. It does not
> necessarily mean (as Scott implies) that this is because the
> uncertainty in data is unreasonably low, but can also mean that there
> are systematic problems with the FEFF calculations that do not account
> for the data as accurately as it can be measured. For most "real"
> data, it is likely that both errors FEFF and a slightly low estimate
> for the uncertainty in the data contribute to making reduced
> chi-square much larger than 1.
>
> And, yes, the community-endorsed recommendation is to report either
> chi-square or reduced chi-square as well as an R-factor. I think some
> referees might find it a little deceptive to report R-factor because
> it is "acceptably small" but not reduced chi-square because it is "too
> big".
More information about the Ifeffit
mailing list