Question about transform windows and statistical parameters
Hello everybody, I am working on fitting some EXAFS of amorphous materials and have noticed an odd (in my mind) behavior when changing transform windows. I settled on a fit using all three k-weights and the Hanning transform window obtaining statistical parameters of R=0.0018 and chi_R=361. I decided to change the transform window to a Kaiser-Bessel to see what would happen. The refined parameters came out more or less the same, well within the error bars, with the Hanning windows having slightly smaller error bars. But my statistical parameters changed significantly to R=0.0022 and chi_R=89.354. It seems that this large change may be related to why we can't use the chi_R parameter to compare fits over different k-ranges, but I am not sure about that. Have other people seen this? I would guess it means that when looking for trends in different data sets, it is more important to be consistent, rather than which specific window type is used. Thanks, Brandon
Hi Brandon:
I have a different idea on what might be happening. Take a look at the
window function and the data in k-space for both window functions. The
kaiser-bessel window works in a different way from the Hanning window. They
emphasize different regions of the data differently. The sill part of the
window is defined differently. Usually use a dk = 1 or 2 A-1 with a Hanning
window. Take a look at the kq data showing the signal q that is used in R
for the fit. The Kaiser-Bessel window works best with dk values of 3 to 4
A-1. This results in a slowly giving more importance to data in the k-range.
Again look at the kq data to see this effect.
If you use a Kaiser-Bessel window with a small dk then the data used in the
fit will extend more (as compared to Hanning) beyond the k-range set by kmin
and kmax. Same for the Hanning window, but with a large dk value as
compared to K-B.
So by simply changing the window without changing dk, you are changing the
information that is Fourier transformed and that results in slightly
different values and all the rest.
If you need an example: I can send you my book chapter :). It has an
excellent section on FT and all the buttons that go with them.
Kelly, S. D., Hesterberg, D. and Ravel, B. (2008). Analysis of soils and
minerals using X-ray absorption spectroscopy. Methods of soil analysis, Part
5 -Mineralogical methods. Ulery, A. L. and Drees, L. R. Madison, WI, USA,
Soil Science Society of America: 367-463.
Cheers,
Shelly
On Wed, May 11, 2011 at 2:46 PM, Brandon Reese
Hello everybody,
I am working on fitting some EXAFS of amorphous materials and have noticed an odd (in my mind) behavior when changing transform windows. I settled on a fit using all three k-weights and the Hanning transform window obtaining statistical parameters of R=0.0018 and chi_R=361. I decided to change the transform window to a Kaiser-Bessel to see what would happen. The refined parameters came out more or less the same, well within the error bars, with the Hanning windows having slightly smaller error bars. But my statistical parameters changed significantly to R=0.0022 and chi_R=89.354. It seems that this large change may be related to why we can't use the chi_R parameter to compare fits over different k-ranges, but I am not sure about that. Have other people seen this? I would guess it means that when looking for trends in different data sets, it is more important to be consistent, rather than which specific window type is used.
Thanks, Brandon
_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Hi Scott and Shelly,
Thanks for the replies.
Scott, I have read a bit about how ifeffit computes the error. I have a
hard time keeping that information handy when I start seeing stuff like
this, but that was the source of my comment about it being related to
compared chi-square with different k-ranges. Thanks for the refresher,
hopefully it sinks in a bit better this time! I tried your suggestion with
epsilon and the chi-square values came out to be very similar values with
the different windows. Does this mean that reporting reduced chi-square
values in a paper that compared several data sets would not be necessary
and/or appropriate? Would setting a value for epsilon allow comparisons
across different k-ranges, different (but similar) data sets, or a
combination of the two using the chi-square parameter?
Shelly, thanks for the tip on appropriate dk values for the two windows. I
would appreciate a copy of your book chapter. Changing the dk value to 2
using the K-B window produced similar chi-square values compared to a
Hanning window with a dk=1.
In playing around with different windows and dk values my fit variables
generally stayed within the error bars, but the size of the error bars could
change more than a factor 2. Does this mean that it would make sense to
find a window/dk that seems to "work" for a given group of data and stay
consistent when analyzing that data group?
Best Regards,
Brandon
On Wed, May 11, 2011 at 2:41 PM, Shelly Kelly
Hi Brandon:
I have a different idea on what might be happening. Take a look at the window function and the data in k-space for both window functions. The kaiser-bessel window works in a different way from the Hanning window. They emphasize different regions of the data differently. The sill part of the window is defined differently. Usually use a dk = 1 or 2 A-1 with a Hanning window. Take a look at the kq data showing the signal q that is used in R for the fit. The Kaiser-Bessel window works best with dk values of 3 to 4 A-1. This results in a slowly giving more importance to data in the k-range. Again look at the kq data to see this effect.
If you use a Kaiser-Bessel window with a small dk then the data used in the fit will extend more (as compared to Hanning) beyond the k-range set by kmin and kmax. Same for the Hanning window, but with a large dk value as compared to K-B. So by simply changing the window without changing dk, you are changing the information that is Fourier transformed and that results in slightly different values and all the rest.
If you need an example: I can send you my book chapter :). It has an excellent section on FT and all the buttons that go with them.
Kelly, S. D., Hesterberg, D. and Ravel, B. (2008). Analysis of soils and minerals using X-ray absorption spectroscopy. Methods of soil analysis, Part 5 -Mineralogical methods. Ulery, A. L. and Drees, L. R. Madison, WI, USA, Soil Science Society of America: 367-463.
Cheers, Shelly
On Wed, May 11, 2011 at 2:46 PM, Brandon Reese
wrote: Hello everybody,
I am working on fitting some EXAFS of amorphous materials and have noticed an odd (in my mind) behavior when changing transform windows. I settled on a fit using all three k-weights and the Hanning transform window obtaining statistical parameters of R=0.0018 and chi_R=361. I decided to change the transform window to a Kaiser-Bessel to see what would happen. The refined parameters came out more or less the same, well within the error bars, with the Hanning windows having slightly smaller error bars. But my statistical parameters changed significantly to R=0.0022 and chi_R=89.354. It seems that this large change may be related to why we can't use the chi_R parameter to compare fits over different k-ranges, but I am not sure about that. Have other people seen this? I would guess it means that when looking for trends in different data sets, it is more important to be consistent, rather than which specific window type is used.
Thanks, Brandon
_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
On Wednesday, May 11, 2011 11:45:43 pm Brandon Reese wrote:
Does this mean that reporting reduced chi-square values in a paper that compared several data sets would not be necessary and/or appropriate?
Heavens! No! That we don't have a reliable way of estimating epsilon says that we cannot apply the standard criterion for recognizing a good fit (i.e. in Gaussian statistics, a reduced chi-square of 1 indicates a good fit). That is, in Ifeffit/Artemis, reduced chi-square for a single fit cannot be interpretted. However, reduced chi-square can be used to assert that one fitting model is an improvement on another fitting model. If reduced chi-square gets significantly smaller, then the second fitting model can be said to be an improvement over the first. So, if the point of a paper is to say that your sample behaves *this* way and not *that* way, one of the tools available to you for making that argument is that the data are more consistent with *this* model because its reduced chi-square is significantly smaller than for *that* model. Of course, reduced chi-square can only be compared for fitting models which compute epsilon the same way or use the same value for epsilon.
Would setting a value for epsilon allow comparisons across different k-ranges, different (but similar) data sets, or a combination of the two using the [reduced] chi-square parameter?
Yup. What you wrote wasn't strictly wrong, but considering the reduced chi-square lets you also compare fits with different variable parameters. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/
Brandon, Scott's answer assumed that epsilon was, in fact, the source of difference between the ratio of R and reduced chi-square, while Shelly's answer assumed that the k-ranges of the window functions was dramatically different. Either (or both) of these could be part of the explanation. I would ask: Were the epsilons reported from the two fits very different?, and What were the full set of parameters for the Fourier window function? It appears from your response that epsilon was fairly different. But you also say that you used a Kaiser-Bessel with dk = 1 or 2. Yikes. These give very poor FT windows. The Kaiser Bessel window should have dk of at least 3 (4 or 5 would be my recommendation), or there is far too sharp a drop at kmin and kmax and much ringing of the chi(R) data. My suspicion would be that this is actually the main cause of the original differences you were seeing. Setting epsilon by hand is a completely reasonable thing to do, if you have a better idea of the noise in chi(k) than Ifeffit's guess. Pulling a number out of thin air might not qualify as a better estimate. ;). Setting epsilon to be the same for several data sets could be OK, but it assumes that the spectra are equally noisy. Again, this may be OK, but if you have to do this because Ifeffit's estimates vary between data sets, than I'd suspect that they are not equally noisy. As Bruce says, the reported reduced chi-square is the statistic that is most informative in deciding if one fit is better than another. Cheers, --Matt
participants (4)
-
Brandon Reese
-
Bruce Ravel
-
Matt Newville
-
Shelly Kelly