[Ifeffit] Reduced chi square values versus F-tests for second shell fits

[Bruce Ravel]

Statistical parameters can be used with any kind of fit.  I would
consider that as a sort of tautology.  Given that a fit is performed
within a particular statistical framework, the tools standard to that
framework are the tools to use.  Our software does not police that
your fitting model is defensible, only that it is expressed correctly
in a numerical sense.  Thus RCS and other statistical parameters are
always well defined.

That said, I am very skeptical of any analysis of only the second
shell.  I explain my reasoning in some detail here:

http://www.mail-archive.com/ifeffit@millenia.cars.aps.anl.gov/msg03563.html

If I understand you correctly, you are doing something similar to what
that other person was doing.  By not including the first shell in the
fit, you are artificially excluding correlations between the two
spectral regions and between the parameters you use to model them.

I have no doubt that you are better able to model the Fourier
components of the second shell in the limited sense of evaluating
misfit when you exclude the first shell from the fit.  You made the
rather arbitrary decision to exclude a major source of uncertainty
from your fitting model.  That does not make your fit more defensible.

Of course, there is always the possibility that I didn't quite
understand the question....

B

http://www.mail-archive.com/ifeffit@millenia.cars.aps.anl.gov/msg03563.html

On Thursday, March 21, 2013 01:05:27 PM Matt Siebecker wrote:
> Dear All,
>  
> Would it be recommended to compare reduced chi square (RCS) values
> from the fit of the second shell only?  While
> fitting the second shell only, the differences between RCS values of
> different models can be greater than when including both the first and
> second shells in the fit.  For example, the RCS values of model 1 and model
> 2 for fits of the first and second shells versus just the second shell are
> 45 and 18, and then 60 and 17, respectively. This to me indicates that
> fitting the second shell separately from the first gave me a slightly
> better description of the misfit in that R-range.  Regardless of the
> fitting range the RCS values say model 2 improves the fit significantly.
>  
> However, if I use an F-test to compare models while fitting the
> second shell only, I’ve found that the difference between the models is
> actually less important than when including both the first and second
> shells in the fit. For examples, the F-test values of model 1 versus model
> 2 for fits of the first and second shell versus just the second shell are
> P=0.047 and then P=0.113, respectively.  This is perhaps caused by
> the small R-range and hence decreased Nind when fitting the second shell
> only. 
> However, it also indicates to me that the F-test says perhaps one
> fitting model is not such an improvement on the fit as I thought it was
> based purely on the decrease in RCS value.  I assume significant
> decreases in RCS values to be about >2x, and P <0.05 for the F-test.
>  
> Should RCS values and F-tests for comparing fitting models only be
> used when fitting two shells together or can they be used when just fitting
> the second shell?
>  
> Kind regards,
> Matt Siebecker


-- 

 Bruce Ravel  ------------------------------------ bravel at bnl.gov

 National Institute of Standards and Technology
 Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2
 Building 535A
 Upton NY, 11973

 Homepage:    http://xafs.org/BruceRavel
 Software:    https://github.com/bruceravel