Hey Carlo, Here is my answer to your other two questions, also CCed to the mailing list.
2. What is the justification for fitting the background during the fitting process. I understand that it is important to remove background large k oscillations but it seems to me that this is better done without the bias of simultaneously fitting FEFF shells. I know that Steve Wasserman and Jeff Terry have a Mathematica background subtraction algorithm which allows you to back FT the region below Rbkg and subtract it from chi. This seems to be roughly the same idea as the background fit but is less biased by the fitting. I would appreciate your comments. Also, is this now a part of FEFFIT or is it solely in Artemis?
It is roughly the same idea, but I would say that you *want* the background to be biased by the fitting. The issue here needs to be restated in terms of what is meant by "a good background removal". Implicit in the back transform algorithm that you describe is the assumption that a good background removal makes the low-R region "look pretty". That is, one assumes that the low-R regions should tail away in some monotonic fashion. Basically, this is an esthetic argument rather than a statistical or scientific argument. On some level, subtracting away a portion of the spectrum via a back transform algorithm is not horribly different from modifying the plot of the chi(R) spectrum with Photoshop or The Gimp. First off, I should explain how the background co-refinement in Ifeffit works. You define a lower bound of the fitting range in R-space. This Rmin value takes the role of Rbkg in the autobk algorithm. That is, Rmin is the cutoff in R-space above which you will use Fourier components from feffNNNN.dat files to fit the data and below which you will use a spline to fit the data. The spline is determined in exactly the same way that the spline in autobk is determined. Is this cheating in terms of number of `independent' point? No. The splin has exactly the number of knots corresponding to the band whose width in R is from 0 to Rmin. That is, different information is used by the spline than by the rest of the fit. Why is another pass of the autobkl algorithm needed at this stage? Why couldn't autobk just do the job correctly up front? Well, this is sort of a numerical issue. Autobk has a really hard job to perform. If you Fourier transform the entire mu(E) spectrum, you will find that the Fourier contribution from the mu0(E) function is much much huger than the contribution from the chi(K) function. Thus autobk is attempting to isolate something small out of something large. By the time Artemis sees the data, mu0(E) is all gone. Further refining the background is thus a matter of separating something small from something small. The REAL REASON to do the background co-refinement is because Ifeffit (and thus Artemis) will report on the **correlations between** the background parameters and the things you are really interested in, such as delta_R and sigma^2. This is improtant. Even more it is a way of defining "goodness" of a background removal that removes esthetic concerns. A "well-removed" background is one that is not too highly correlated with the fitting parameters. Of course, "not to highly" is a rather subjective, but it atleast is a number to work with. So what is the right value of Rmin? Well, you want to choose an Rmin that is high enough to make a pretty picture (esthetics is not entirely without merit, after all) but low enough so as not to introduce excessive correlation with the fitting parameters. Rmin, therefore, is one of the parameters that you need to fiddle with in your data. For oxides (and carbides, nitrides, etc) Rmin might be quite low. Indeed, there may not be a value which gives a spline that is loosely correlated with the fitting parameters and which also satisfies the esthetic criteria of a pretty low-R region taht tails off monotonically. To avoid harming the data by excessive correlation with the spline, you may be forced to accept low-R peaks even in your published data. Finally, how does this discussion relate to the background removal done in Athena? Well, it should be clear by now that you cannot know the right value of Rmin a priori. You have to do some fits and measure the correlation between the background and the fitting parameters to judge what Rmin should be. Good practice with Athena is to choose a rather low value of Rbkg and not worry that some icky little low-R peaks might remain in the data. If you choose a low Rbkg in Athena, you can always choose a higher Rmin value in Artemis. But you cannot go the other way. You should not choose a value of Rmin that is *lower* than the Rbkg used in Athena because then you will not correctly evaluate the correlations.
3. When it comes time to report results, is it better to be able to subtract the background? Is it legitimate to report fitting parameters without including the "hidden" background parameters?
I say yes, it is better to subtract the background found by Artemis. (That's what the "use previous background" button does.) Esthetics, as I confessed above, is still important. And yes, reporting fitting parameters from a fit with a background co-refinement is most certainly legitamate. Indeed, it is more so because you can also report the level of correlation with the background spline. This gives you reader MORE confidence in your results and a better ability to interpret their validity. Sorry I was so long winded, but these are subtle issues. When I give a series of lectures on data analysis, I usually spend about 30 minutes discussing background issues. HTH B PS: Background co-refinement is, like everything else mathematical, something done by Ifeffit. Ifeffit does all the heavy lifting, Artemis just keeps it all organized. -- Bruce Ravel ----------------------------------- ravel@phys.washington.edu Code 6134, Building 3, Room 222 Naval Research Laboratory phone: (1) 202 767 5947 Washington DC 20375, USA fax: (1) 202 767 1697 NRL Synchrotron Radiation Consortium (NRL-SRC) Beamlines X11a, X11b, X23b, X24c, U4b National Synchrotron Light Source Brookhaven National Laboratory, Upton, NY 11973 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/
Hi all, My two cents, FWIW: I prefer not to use the background found by Artemis in final fits (well, actually I'm still using FEFFIT, but I think the issues Bruce discussed haven't changed). The correlations reported by IFEFFIT or FEFFIT, while very useful, only really probe the area of fitting space right around one pair of variables. In complex fits, there are often multiple minima in fitting space, or nonlinear effects that involve more than one set of variables. Just because a background subtracted during the fitting process doesn't report a high correlation doesn't necessarily mean that it hasn't influenced the fit. I came up again this issue when I was doing multiple-dataset fits on a temperature series; if I let the background refine in this way for the spectrum at each temperature, I sometimes destroyed temperature trends in things like lattice parameters that showed up perfectly well without the second background subtraction...and the effect of the background did not show up in the reported correlation coefficients. In terms of the "not cheating" argument (the number of knots is the same as the number of independent points), I need to be convinced that it is really OK on data that has already been through Athena. After all, haven't two splines been subtracted from the data? So why doesn't that gobble up twice as many of the independent points? Currently, I do use the background feature during the fitting process, because it does help guide me as to where rbkg should go in the way Bruce describes. But then I run AUTOBK (basically the routine used by Athena) with the rbkg at the value I've determined is appropriate and leave it at that. That way, I'm comfortable that rbkg has been chosen in some rational, defensible way (the big selling point for using Artemis to further refine the background), but I don't have to think about what it means to use a two-pass background subtraction. --Scott Calvin Naval Research Lab Code 6344 --
Hi, I figured I should comment on some of the issues on background subtraction. I agree with 99.9% of what Bruce said. This probably represents the UWXAFS/Ifeffit party line, and so it's definitely worth questioning. Much about this topic is not completely known, and I'm certainly open for suggestions... Carlo mentioned an algorithm:
I know that Steve Wasserman and Jeff Terry have a Mathematica background subtraction algorithm which allows you to back FT the region below Rbkg and subtract it from chi. This seems to be roughly the same idea as the background fit but is less biased by the fitting. I would appreciate your comments.
I don't know the details of Steve and Jeff's method, but it sounds close to Autobk/spline(). Two other methods are also similar: 1) the method of Cook and Sayers, which took mu(E) and smoothed it to give the background function, and 2) the method of Li, Bridges, and Booth (in their work on Atomic XAFS), where they subtracted a first shell Feff calculation for chi(k) from the mu(k) and then filtered out the high frequency parts to reveal structure in the background. There may be other similar methods or variations as well. These all share the idea that mu0(E) is defined by the lowest frequency components of mu(E). The subtleties are then 1) whether you use the higher-frequency components in the fit for the background (as most 'classic' background subtractions and Cook and Sayers do), 2) whether you use an estimate of the first shell EXAFS, and 3) how you control the flexibility of the spline. Of course, the main issue with background subtraction in EXAFS is that the background extends into the first shell, and may affect the results for the first shell parameters. This is a "well-known" problem in EXAFS, though perhaps more widely assumed and accepted than actually studied. There's also the related idea that background subtraction should leave a "pretty" chi(k) and chi(R). Again, I think this not as well-established as we might hope, but is probably reasonable. For now, I'll assume it's right.... If the background "leaks into" the first shell (as everyone knows it does), then the first shell probably also extends to the low-R "background region". Certainly, if there were no overlap of the first shell and background, then any old background subtraction method will work just fine. This suggests that we *should* take the first shell into account when determining the background. I heartedly agree with Bruce on this (and most other things). How this happens is not necessarily obvious. In Autobk/spline(), the first shell leakage to low-R can be taken into account crudely by supplying a 'standard': a chi(k) spectra with approximately the right first shell. The idea is that the first shell of the standard can be scaled in amplitude to match the first shell of the data and then predict the leakage of the first shell to low-R. Without a 'standard', Autobk/spline() simply minimizes the low-R components, which ignores the first shell leakage. Li, Bridges, and Booth sort of boot-strapped the solution (first using a simple background to get a fitted chi, then subtracted that from mu to get a refined background). Refining the background in Feffit/feffit() is roughly the same concept, but allows a better mode for the first shell. It also allows us to assess correlations between background and structural parameters, which seems useful. Another important difference is that Autobk/spline() use a very different FT range than Feffit/feffit(), so low-R 'ugly bumps' that were reduced in Autobk/spline() might appear with other FT settings. The ability to assess the correlations of the background and structural parameters was a main motivation for putting this feature in Feffit. Everyone "knew" that you can get "wrong answers" for the first shell parameters because the background was "wrong", but it was rare to measure how wrong the first shell parameters could be. By refining background parameters with structural parameters, we can now measure it easily. In general, the background parameters and the structural parameters seem to be slightly correlated, with E0 and R1 influenced more than N and sigma2. Some people are uncomfortable with the idea of measuring background parameters twice, which is what running Autobk/spline() and then refining the background with Feffit/feffit(). I don't see a problem with this from an 'information theory' point of view. There are a fixed set of parameters set aside to account for the background. Autobk/spline() measures them and the Feffit/feffit() refines that measurement. More simply, if a parameter can be measured, it is allowed to be measured. But that presents a good opportunity to bring up the bkg_cl() command in Ifeffit. By itself, bkg_cl() gives a pretty bad background, though it does give a reasonably good and stable normalization. Importantly, it does no spline fit at all, and so "uses up" no independent points (if you're hung up on counting such things). Doing this, and then including the background in feffit() really does put the refinement of all variables in one fit. Many (possibly most) people greatly prefer refining everything at once. It's easy to do (athena even has a pull-down menu and guesses the central atom for you!!), and a good thing to try if you're worried about the relation between background subtraction and fitted parameters. --Matt
participants (3)
-
Bruce Ravel
-
Matt Newville
-
Scott Calvin