[Ifeffit] Chi^2 and R-factor reporting requirements

[Bruce Ravel]

On Monday 12 July 2004 03:25 pm, Scott Calvin wrote:
> Having said that, I'd love a discussion as to how people obtain
> meaningful estimates of epsilon, or under what conditions they
> consider the Ifeffit default good enough.
>
> As you noted, it is worth reiterating that the algorithm Ifeffit uses
> to find uncertainties in fitted parameters is independent of the
> choice of epsilon.

I would emphasize that last point.  The error bars on parameters are
always reasonable because they are computed by scaling the diagonal
elements of the covarience matrix by the square root of chi-square.
That is *the same thing as* reeveluating the covarience matrix with
the value of epsilon that results in a reduced chi square of 1.  Thus,
if you believe that the fit is a good fit, the error bars are the
right size.

As far as the actual value of epsilon -- I am not sure it is really
all that important.  There is plenty of information available to your
to evaluate a fit.  The R-factor is a start, as are the error bars and
ones physical intuition about the system.

The use of reduced chi-square is to compare two different models.  If
you change a fitting model in some way, you expect the reduced
chi-square to be sufficiently smaller for the better fit.  If reduced
chi-square doesn't change enough, then the two fitting models are
statistically indistinguishable.

Another way of saying that is that the R factor is a useful metric for
deciding if THIS fit is reasonable.  The reduced chi-square is a
useful metric for deciding if THIS fit is better than THAT fit.

As to whether you need to report reduced chi-square in a paper, I
agree with Matt.  If it's an important part of the story, it's an
important part of the paper.  But if you include it in the paper, you
had better be prepared to explain the previous paragraph and to
explain why it is so hard to estimate epsilon.  Or at least to refer
to a paper which explains those things to your satisfaction.

B


-- 
 Bruce Ravel  ----------------------------------- ravel at phys.washington.edu
 Code 6134, Building 3, Room 405
 Naval Research Laboratory                          phone: (1) 202 767 2268
 Washington DC 20375, USA                             fax: (1) 202 767 4642

 NRL Synchrotron Radiation Consortium (NRL-SRC)
 Beamlines X11a, X11b, X23b
 National Synchrotron Light Source
 Brookhaven National Laboratory, Upton, NY 11973

 My homepage:    http://feff.phys.washington.edu/~ravel 
 EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/