RE: [Ifeffit] R-factor
Hi Alison, I agree entirely with Victor and Anatoly. In addition, I'd say that in my work an R-factor of above 0.1 is pretty sketchy. How sketchy depends on the complexity of the fit, of course; you're going over a big r-range, so if it takes you only two or three parameters to get an R-factor of 0.11, I guess that's not too bad. But if you have more parameters than that and it's still not fitting so well, I'd be concerned. As a referee, for example, that would be a much larger warning flag to me than the little discrepancy between the R-factors for the k- and q-space fits. Just to echo Victor and Anatoly again: it's quite easy for me to believe that the fitting algorithm just happened to find a somewhat different minimum in the fitting space in the two cases. They both suggested ways to test if that is the case. Although I've never fit in q-space, I've seen something that on the face of it is equally baffling: sometimes I've added an additional fitting parameter, changing nothing else, and seen the R-factor <italic>increase </italic>slightly. In some ideal sense, that shouldn't be--the fitting routine (this one happened to be Ifeffit) "should" at worst get the identical fit as to when the new parameter was constrained. But it is impossible to write a fitting routine that is guaranteed to find the closest possible fit, so all routines will occasionally generate those kinds of results. --Scott Calvin Sarah Lawrence College
-------------- Original message --------------
Bruce,
I've looked, and my R-factors for the back-transformed
space are not necessarily twice the value of the R-factors
for the k-space fits. In fact, they are often quite
close. For example, just recently I had a k-space
R-factor of 0.102 and a q-space R-factor of 0.113. Now, I
realize those numbers are very close, but I'm afraid if I
try to publish this, then I will get criticism for the
q-space R-factors being larger. If I can explain it, then
maybe it won't be a problem.
Thanks to all for taking the time to discuss my concerns. To clarify, both fits were done with exactly the same parameters. So, I believe Bruce's description (below) best explains what is happening. I also realize that my R-factors are rather large compared to the norm for very well-ordered systems. I am, however, fitting protein data that are characteristically not well-ordered. Thanks again for your time, concerns, and suggestions. They were all much appreciated. alison Alison Costello University of New Mexico MSC03 2060 1 University of New Mexico Albuquerque, NM 87131
Viktor and Anatoly are most certainly correct that you parameters should be consistent between the k and q fits. That is probably the bottom line in this discussion.
But as long as we are talking about R-factors....
In the example you give here, it seems to me that you are seeing the interplay of two different aspects of the calculation. In the k-space R factor, the R-factor includes high frequency Fourier components -- both structural and noise. (That is, the R factor considers differences between data and theory and the data includes high frequency pieces in the k-space fit.) The q-space fit has had the high frequency stuff removed, but has the factor of two given that it's a complex function. Those two competing parts of the calculation should explain away the values you quote.
B
-- Bruce Ravel ----------------------------------- bravel@anl.gov -or- ravel@phys.washington.edu Environmental Research Division, Building 203, Room E-165 Argonne National Laboratory phone: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793
My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/
Hi Bruce and Matt, just thinking while fitting: wouldn't it be possible-better-auspicable-desirable to have a simultaneous fit in both R and k space and evaluate the statistics of the two fits in some intelligent way? Just a though Stefano -- ____________________________________________ Stefano Ciurli Professor of Chemistry Laboratory of Bioinorganic Chemistry Department of Agro-Environmental Science and Technology University of Bologna Viale Giuseppe Fanin, 40 I-40127 Bologna Italy Phone: +39-051-209-6204 Fax: +39-051-209-6203 "Fatti non foste a viver come bruti, ma per seguir virtute e canoscenza" Dante Alighieri - Inferno - Canto XXVI "Ihr seid bestimmt, nicht Tieren gleich zu leben, Nein, Tugend zu erringen und Erkenntnis" "Ye were not form'd to live the life of brutes, But virtue to pursue and knowledge high"
Hi Stefano, I'm not sure what you're suggesting--a real simultaneous fit, which somehow weights the fit in R-space and k-space to arrive at a single set of parameters? Or just two fits in series so that you only hit one button? In the first case, I'm not sure I see a big benefit considering that the weighting scheme becomes in effect another opaque parameter that can be used to generate arguments among experts. :) In the second case, it seems to me more straightforward just to run the fit twice. But maybe I'm not understanding your suggestion? --Scott Calvin Sarah Lawrence College At 04:17 PM 7/8/2005 +0200, you wrote:
Hi Bruce and Matt, just thinking while fitting: wouldn't it be possible-better-auspicable-desirable to have a simultaneous fit in both R and k space and evaluate the statistics of the two fits in some intelligent way? Just a though Stefano -- _>
Hi Scott,
I'm not sure what you're suggesting--a real simultaneous fit, which somehow weights the fit in R-space and k-space to arrive at a single set of parameters? Or just two fits in series so that you only hit one button?
first of all let me tell you that the idea of running the fit in R-space somehow disturbs my simple mind, considering that the R-space depends on the processing parameters (window type and width, weight etc). So, I would assume that a fit in k-space would be auspicable. Then, I discovered that EXCURV, the competing program at least for biological XAS, performs the fit BOTH in R-space and in k-space, and I am pretty sure that the parameters it gets are slightly different. then, yes, I was thinking of two different fits hitting the same button (let's say in Diana - or Artemis - instead of having just the option of running the fit either in R- or in k-space, one could have an additional button telling the program to run both fits)
In the first case, I'm not sure I see a big benefit considering that the weighting scheme becomes in effect another opaque parameter that can be used to generate arguments among experts. :)
that I understand, not being an expert... :-)
In the second case, it seems to me more straightforward just to run the fit twice.
OK. probably yes, but if Bruce may think of a slight improvement ... it was just an idea Regards, Stefano -- ____________________________________________ Stefano Ciurli Professor of Chemistry Laboratory of Bioinorganic Chemistry Department of Agro-Environmental Science and Technology University of Bologna Viale Giuseppe Fanin, 40 I-40127 Bologna Italy Phone: +39-051-209-6204 Fax: +39-051-209-6203 "Fatti non foste a viver come bruti, ma per seguir virtute e canoscenza" Dante Alighieri - Inferno - Canto XXVI "Ihr seid bestimmt, nicht Tieren gleich zu leben, Nein, Tugend zu erringen und Erkenntnis" "Ye were not form'd to live the life of brutes, But virtue to pursue and knowledge high"
On Tuesday 12 July 2005 04:05, Stefano Ciurli wrote:
I'm not sure what you're suggesting--a real simultaneous fit, which somehow weights the fit in R-space and k-space to arrive at a single set of parameters? Or just two fits in series so that you only hit one button?
first of all let me tell you that the idea of running the fit in R-space somehow disturbs my simple mind, considering that the R-space depends on the processing parameters (window type and width, weight etc). So, I would assume that a fit in k-space would be auspicable. Then, I discovered that EXCURV, the competing program at least for biological XAS, performs the fit BOTH in R-space and in k-space, and I am pretty sure that the parameters it gets are slightly different.
then, yes, I was thinking of two different fits hitting the same button (let's say in Diana - or Artemis - instead of having just the option of running the fit either in R- or in k-space, one could have an additional button telling the program to run both fits)
My US$0.02 worth: Conceptually, corefining in k and R space is similar to corefining multiple k-weights. That is, it is corefining two different perspectives on the same data set, thus adding more points to the evaluation of chi-square without changing the information content of the data. The multiple k weight fit provides a clear benefit. The different k-weightings affect different parameters differently. k^3 enhances, in a sense, the sensitivity of the fit to delta_r relative to delta_e0 while k^1 does the reverse. Doing a 1,3 fit is a sort of compromise in hopes of partially disentangling the correlations between those parameters. I don't see the similar advantage to doing a k,R fit. That is not to say there isn't one, its just says that the merit of doing so has not yet percolated into my tiny brain. I just don't see how a k,R fit will serve to disambiguate some part of the fit that is ambiguous in a k-only or R-only fit. B P.S. I would be surprised if the choice of k-space window function has an effect on the fitting parameters outside their uncertainties so long as a modest value for dk is used. -- Bruce Ravel ----------------------------------- bravel@anl.gov -or- ravel@phys.washington.edu *** My cell phone number has changed. Please ask if you need the new number Environmental Research Division, Building 203, Room E-165 Argonne National Laboratory phone and voice mail: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/
Hi Bruce and Matt, Ok for all your explanations. Thanks! Stefano -- ____________________________________________ Stefano Ciurli Professor of Chemistry Laboratory of Bioinorganic Chemistry Department of Agro-Environmental Science and Technology University of Bologna Viale Giuseppe Fanin, 40 I-40127 Bologna Italy Phone: +39-051-209-6204 Fax: +39-051-209-6203 "Fatti non foste a viver come bruti, ma per seguir virtute e canoscenza" Dante Alighieri - Inferno - Canto XXVI "Ihr seid bestimmt, nicht Tieren gleich zu leben, Nein, Tugend zu erringen und Erkenntnis" "Ye were not form'd to live the life of brutes, But virtue to pursue and knowledge high"
Hi Stefano, On Fri, 8 Jul 2005, Stefano Ciurli wrote:
Hi Bruce and Matt, just thinking while fitting: wouldn't it be possible-better-auspicable-desirable to have a simultaneous fit in both R and k space and evaluate the statistics of the two fits in some intelligent way? Just a though Stefano
Trying fits that simultaneously uses R- and Q-space is an OK idea, and like Bruce says, isn't very different from using multiple k-weights. I think it would be really interesting to fit in the "R+Q" space generated by the wavelet transforms. For now, I sort of doubt that fitting in R-space and Q-space simultaneoulsy will make a huge difference to fitting in R-space alone, but I'd be curious to see evidence either way. On the other hand, fitting unfiltered k-space is always a bad idea. Using R-space and Q-space allows you to select the k- and R-range of your data to model, which you want. This also allows a proper error analysis. Using unfiltered k-space does not allow this as there are systematic signals (the higher shells) in the data that are not modelled. Fitting in unfiltered k-space also gives erroneous R-factors, as was recently discussed here. Cheers, --Matt
Hi Matt, Thinking about problems. You once told me that:
Fitting in unfiltered k-space also gives erroneous R-factors, as was recently discussed here.
would you give me a direction to the discussion about this point that I can read? Thanks Stefano -- ____________________________________________ Stefano Ciurli Professor of Chemistry Laboratory of Bioinorganic Chemistry Department of Agro-Environmental Science and Technology University of Bologna Viale Giuseppe Fanin, 40 I-40127 Bologna Italy Phone: +39-051-209-6204 Fax: +39-051-209-6203 "Fatti non foste a viver come bruti, ma per seguir virtute e canoscenza" Dante Alighieri - Inferno - Canto XXVI "Ihr seid bestimmt, nicht Tieren gleich zu leben, Nein, Tugend zu erringen und Erkenntnis" "Ye were not form'd to live the life of brutes, But virtue to pursue and knowledge high"
On Monday 19 September 2005 07:31, Stefano Ciurli wrote:
Hi Matt,
Thinking about problems. You once told me that:
Fitting in unfiltered k-space also gives erroneous R-factors, as was recently discussed here.
would you give me a direction to the discussion about this point that I can read?
Stefano, I don't know if there is an explicit reference. Just consider the functional form of the R-factor. It's essentially a measure of misfit -- a quantitative measure of how much the red line falls on top of the blue line. Consider a first shell fit to copper metal. In R space and under the first peak, the red line falls right on top of the blue line and the R-factor is small. In k space, the red line only has the frequencies of the first shell while the data has all frequencies. In this space, the red line doesn't look much like the blue line, even for an excellent fit. Consequently the R factor is quite large. That is, I think, what Matt meant by "erroneous" -- not that I can claim to speak for him! ;-) B -- Bruce Ravel ----------------------------------- bravel@anl.gov -or- ravel@phys.washington.edu Environmental Research Division, Bldg 203, Room E165 Argonne National Laboratory phone: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/
Hello everybody,
Thinking about problems. You once told me that:
Fitting in unfiltered k-space also gives erroneous R-factors, as was recently discussed here.
would you give me a direction to the discussion about this point that I can read?
Pardon my poor knowledge of EXAFS fitting and reliability of the results, but more generally, I wondered if EXAFS fitting in raw unfiltered k-space is always relevant. Let me explain it; I do agree that in certain cases, when the whole structure is known and all EXAFS contributions well modelled by an ab-initio model (e.g., feff), you may well reproduce all oscillations of the EXAFS spectrum, and then fitting in raw k can be OK. However, more often than wanted, the fitting is done with a structural model including only the first few shells. With these shells, we model not only the actual contributions from the neighboring shells, but also that from the more distant shells. Physically, it can be demonstrated that this would not be a problem if the EXAFS signal had a constant amplitude and would spread over an infinite range in k space. However: (a) EXAFS contributions have not a constant amplitude, and (b) they can be recorded over only a few Å-1. So maybe (or even surely, I daresay), the contribution from non-modelled distant shells would affect the structural parameters from the modelled shells. Finally, another issue - for which I would not lay out my neck, though - is the noise. In EXAFS, the signal to noise increases with k, and of course fitting in the raw k space is another way of dealing conveniently with the noise -or relative uncertainty, as we may name it. However, the FT is a way of filtering out some of this noise, and so maximise the signal from a shell. Likewise, fitting in filtered q-space may yield more accurate results, because some of the noise is filtered out. However, I concur that somehow the fact that the uncertainty on the high-q part of the filtered contribution os greater than in the low q-part should be implemented somehow. So there are my own reflexions. Maybe some people have already worked on that, and there are some publications that I've missed (shame on me!). If that's the case, I would welcome any insight. Best regards, Michel Schlegel -- Michel Schlegel Commissariat à l'énergie atomique CEN de Saclay, DEN/DANS/DPC/SCP/LRSI Bat 391 - Piece 205B F91 191 Gif-sur-Yvette Cedex, France Ph: +33 (0)1 69 08 93 84 Fax: +33 (0)1 69 08 54 11
On Tuesday 20 September 2005 02:59, Michel Schlegel wrote:
So maybe (or even surely, I daresay), the contribution from non-modelled distant shells would affect the structural parameters from the modelled shells.
I daresay "surely" as well ;-) Although I prefer to think about the EXAFS problem in R space, I don't think this is the reason to do the fit in R space. Regardless of which fitting space you use, you are relying on Feff to supply some fraction of all the Fourier components in the data. The first-shell feff path only has Fourier components (or frequencies, if you prefer) corresponding to the first-shell portion of chi(R). Thus that feff path can only fit those frequencies. That's true in k-space as well as in R-space What's more, I think the problem of non-modelled shells affecting the modelled portion of the data exists in R-space just as much as in k-space. Because of sigma^2 and the finite data range, peaks have width. In R-space, a given path is centered at a particular R-value (or, in other words, its contribution to the spectrum is dominated by a particular frequency). However, the peak has width (it contains frequencies below and above the dominant frequency). Thus longer paths always interfere with shorter paths to some extent. B -- Bruce Ravel ----------------------------------- bravel@anl.gov -or- ravel@phys.washington.edu Environmental Research Division, Bldg 203, Room E165 Argonne National Laboratory phone: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/
Hello Bruce,
Although I prefer to think about the EXAFS problem in R space, I don't think this is the reason to do the fit in R space. Regardless of which fitting space you use, you are relying on Feff to supply some fraction of all the Fourier components in the data.
We could also used reference-derived phase and amplitude function, as in the good ol'time ;)
The first-shell feff path only has Fourier components (or frequencies, if you prefer) corresponding to the first-shell portion of chi(R). Thus that feff path can only fit those frequencies. That's true in k-space as well as in R-space
True in R-space, and that's why Athena-Ifeffit are doing such a wonderful job at providing windows to fit only selected area of the fourier transform. Almost true also in q-space. In k-space... hmmm, well, I would love to see the mathematical demonstration of it for a limited k-range and a Feff(k) (eventhoug I agree that many people have performed tha fit in k- and R-spaces, and found about the same paramters, so it must be verified somehow).
What's more, I think the problem of non-modelled shells affecting the modelled portion of the data exists in R-space just as much as in k-space. Because of sigma^2 and the finite data range, peaks have width. In R-space, a given path is centered at a particular R-value (or, in other words, its contribution to the spectrum is dominated by a particular frequency). However, the peak has width (it contains frequencies below and above the dominant frequency). Thus longer paths always interfere with shorter paths to some extent.
I think the effect of interfering contributions in the Konigsberger and Prins. What I noticed, though, is that the fit of closer shells can affect that of the furthest shell because of the "spillof" from the first peak. The recciprocal I was not aware of. But I do agree with you on the whole. However, I think that in the FT case, these interences are limited to the nearest shells, whereas in k-space, nothing protects you from shells at, say, 2R and 3R (so to speak. Best regards, Michel -- Michel Schlegel Commissariat à l'énergie atomique CEN de Saclay, DEN/DANS/DPC/SCP/LRSI Bat 391 - Piece 205B F91 191 Gif-sur-Yvette Cedex, France Ph: +33 (0)1 69 08 93 84 Fax: +33 (0)1 69 08 54 11
At 03:07 PM 9/20/2005 +0200, you wrote:
I think the effect of interfering contributions in the Konigsberger and Prins. What I noticed, though, is that the fit of closer shells can affect that of the furthest shell because of the "spillof" from the first peak. The recciprocal I was not aware of. But I do agree with you on the whole. However, I think that in the FT case, these interences are limited to the nearest shells, whereas in k-space, nothing protects you from shells at, say, 2R and 3R (so to speak.
As part of a larger unpublished study that keeps sitting on my back-burner (one of these days maybe I'll finish up the paper and send it somewhere!), I looked at the seriousness of the "leakage" (or spill-over or interference or whatever you want to call it) from higher-R shells in a first-shell fit of fcc metals. Although I don't have the results at my fingertips right now, my recollection is that it was sufficient to completely screw up the dependence of lattice parameter and sigma2 on temperature for temperatures below room temperature. The fact that I've heard occasional assertions that EXAFS is not sufficiently sensitive to accurately yield the thermal expansion of fcc metals below room temperature suggests to me that some people may be underestimating the effect of this leakage. When the outer shells are accounted for (even crudely--no new free parameters are necessary), I found it relatively easy to get the right expansion characteristics. Having said that, of course the problem of leakage is LESS for fits in R-space than k-space, right? At least we're filtering out PART of the signal from outer paths when we do a fit on a portion of R-space, but that's not possible when we do a k-space fit. --Scott Calvin Sarah Lawrence College
Hi Michel, On Tue, 20 Sep 2005, Michel Schlegel wrote:
Pardon my poor knowledge of EXAFS fitting and reliability of the results, but more generally, I wondered if EXAFS fitting in raw unfiltered k-space is always relevant.
I would not call your knowledge of EXAFS fitting "poor". I think you've pretty much got every point exactly right. For the original question (why prefer R-space v. k-space or even Q-space for fitting), the difference is only important when there are spectral content (k- and R-range) that you want to ignore. The content you'd want to ignore is most commonly the high-R shells that you don't (yet?) have a model for. That is, if your model could account for all spectral features, fitting in k- and R-space would be equivalent. For the more common case in which the model is more limited than the data, using R-space makes it very easy to specify which components to ignore. Fitting in Q-space is nearly identical to R-space, expect for an issue you point out later.... Being able to limit the spectral content in this way is entirely to get a good measure of the fit statistics. When fitting in original k-space, you cannot say "fit the first shell and ignore the fact that I'm not modelling the second shell". As Michel points out, the limited k-range and the physics of EXAFS does mean there is 'spectral bleeding', so that the frequencies (R-values) for a single 'shell are not perfectly sharp. Looking at any plot of |chi(R)| it is pretty obvious that the peaks are not delta functions, or even particularly sharp. As Michel put it "the contribution from non-modelled distant shells would affect the structural parameters from the modelled shells". This is unavoidable, but is also generally a small effect. The best things to do are to be aware of this possibility and try to model any significant further shells than those you're really willing to say you've got right. This is why we prefer R-space to Q-space and the older approach of 'Fourier Filtering'. It is often difficult and sometimes impossible to really isolate the 'First Shell' from the 'Second Shell', which can make it dangerous to compare isolated 'First Shells' from systems with different 'bleeding' of higher shells. Michel also wrote:
Finally, another issue - for which I would not lay out my neck, though - is the noise. In EXAFS, the signal to noise increases with k, and of course fitting in the raw k space is another way of dealing conveniently with the noise -or relative uncertainty, as we may name it. However, the FT is a way of filtering out some of this noise, and so maximise the signal from a shell. Likewise, fitting in filtered q-space may yield more accurate results, because some of the noise is filtered out. However, I concur that somehow the fact that the uncertainty on the high-q part of the filtered contribution os greater than in the low q-part should be implemented somehow.
The FT does not actually filter out noise. It can be used to filter out the highest frequencies, and so the sharp spikey spectra that oftens shows up in k-weighted chi(k) at high k (where the noise is larger than the signal). Sadly, this is the part of the noise we care the least about: there is also noise with the same frequency (hopefully lower amplitude!) as the signals we're analyziing. That's not removed by ignoring the high frequency components. In general, I would say that fitting in R-space is slightly preferred over fitting in Q-space and much preferred over fitting in k-space. The only real reason it is much preferred to k-space fitting is that you can systematically ignore shells that you are not modelling. Cheers, --Matt
participants (7)
-
Alison L Costello
-
Bruce Ravel
-
Bruce Ravel
-
Matt Newville
-
Michel Schlegel
-
Scott Calvin
-
Stefano Ciurli