[Ifeffit] measurement uncertainty

[Bruce Ravel]

Gustavo de Medeiros Azevedo said:


 GdMA> I''ve recently started using IFEFFIT, and I'm a bit confused on
 GdMA> the way it estimates the error bars.  In the IFEFFIT Reference
 GdMA> guide, (page 38) you mention the variables epsilon_k and
 GdMA> epsilon_r, where the measurement uncertainty is stored.  In
 GdMA> that same page, you remark that changing the values of
 GdMA> epsilon_k and epsilon_r will change chi_squared, Chi-reduced
 GdMA> but won't affect R_factor and the error bars.  As far as I
 GdMA> understand, the error bars should given by the diagonal
 GdMA> elements of the co-variance matrix, multiplied by the square
 GdMA> root of Chi_reduced.  In this way, I would expect that varying
 GdMA> epsilon_k or epsilon_r should affect the estimated
 GdMA> uncertainties.

 GdMA> Could you, please, clarify this point?

Gustavo,

Well, I live one time zone to the east of Matt, so I guess I'm seeing
this before he does ;-)  I'll take a stab at your question.

epsilon_r is measured from the data by performing the Fourier
transform of chi(k) and then computing the root mean square value of
the magnitude of chi(R) between 15 and 25 Angstroms.  The assumption
is that, because of disorder and mean-free-path effects, any finite
spectral content in that R range can only be due to white noise in the
data.

The relationship between epsilon_k and epsilon_r is established by the
normalization of the Fourier transform.

Because epsilon_R enters into the fitting metric, chi-square, when the
fit is done in R-space (or epsilon_k for a fit in k space), you are
correct that the value of chi-square and reduced chi-square are
affected by the value of epsilon_r.  The R_factor is just a percentage
misfit and so is independent of epsilon.  The error bars are
independent of epsilon as reported by ifeffit, but the reason for this
requires some explanation.

The first question to ask at this point is whether it is valid to
presume that the dominant error in the exafs measurement is the shot
noise measured from the high-R portion of the Foruier spectrum.  In
most situations it is not -- that is, in most situations the shot
noise is much smaller that detector non-linearity, sample
inhomogeneity, errors in the theoretical fitting standards, errors in
the empirical fitting standards, or a whole host of other problems you
might have in a real experiment.  Particularly at 3rd generation light
sources, photon flux and the shot noise associated with it is the
least of your problems.  By a long stretch.

Thus, the way ifeffit uses to estimate epsilon is manifestly wrong.
It is way too small, thus chi-square is very large.  Even for a fit
that is clearly a good fit, such as the common example of a copper
foil that comes with the Artemis, the reduced chi-sqare is much larger
than 1.  By the formalism of Gaussian statistics, we expect that a
good fit has a reduced chi-square of about 1.  So what's going on?

Well, the problem is that ifeffit makes no attempt (since it operates
within a Gaussian framework and not a Baysian framework) to estimate
all the other sources of error besides shot noise.  Thus it is almost
always doomed to have reduced chi-square much larger than 1 (except in
the odd case when the shot noise is quite large).

So what about the error bars?  The diagonal elements of the covarience
matrix (i.e. the error bars) are much too small for the same reason
that the reduce chi-square is too large, i.e. because epsilon has been
vastly underestimated.

Ifeffit then makes the assumption that the fit it just finished was,
in fact, a good fit.  It assumes that the reason that reduced
chi-square is not near one is NOT because it is a bad fit, but rather
because epsilon has been significantly underestimated.  Multiplying
the error bars by the square root of reduced chi-square is
mathematically equivalent to re-evaluating the covarience matrix with
epsilon set to the value it must be for reduced chi-square to equal
1.  Thus, it is the mathematical equivalent of assuming that the fit
is, in fact, a good fit.

So finally, you asked how setting epsilon to some other value will
effect the fit.  The answer is "Not much."  It will have some small
effect in how the fit is evaluated numerically because chi-square may
be of a different order of magnitude.  However, because Ifeffit
rescales the error bars under the assumption that the fit was good,
manually changing epsilon will have little effect on the results.

One more thing needs to be said.  Because Ifeffit *always* assums that
the fit is a good fit and reports statistics accordingly, it is the
RESPONSIBILITY OF THE USER to actually evaluate the quality of the
fit.  Are the best fit values physically reasonable?  Are the
parameters highly correlated?  Are the rescaled error bars of a
reasonable size?  Ifeffit cannot answer those questions.  You must.
And that, finally, is why a human needs to do data analysis.  Sadly we
cannot train a computer or a monkey to decide if the fit is, in fact,
the one you want to publish.

Hope that helps,
Bruce



-- 
 Bruce Ravel  ----------------------------------- ravel at phys.washington.edu
 Code 6134, Building 3, Room 222
 Naval Research Laboratory                          phone: (1) 202 767 5947
 Washington DC 20375, USA                             fax: (1) 202 767 1697

 NRL Synchrotron Radiation Consortium (NRL-SRC)
 Beamlines X11a, X11b, X23b, X24c, U4b
 National Synchrotron Light Source
 Brookhaven National Laboratory, Upton, NY 11973

 My homepage:    http://feff.phys.washington.edu/~ravel 
 EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/