Logicused in Artemis to do the error minimization
Dear All, After the final fitting in Artemis, we get the (i) total error; (ii) the final value of the parameters; (iii) the error in the parameters as well as (iv) the dependencies among the parameters. What is kind of logic used in Artemis to calculate these values? Is it Genetic optimization procedure used in this process? Can anyone help me by giving some references. I was going through the old posts and found two posts. One is this one: http://cars9.uchicago.edu/ifeffit/FAQ/FeffitModeling and the other one mentioned in the same. I am sorry to ask for some help which is not directly related to the XAFS. Thanks in advance. With regards, Pralay. --- Research Student. Solid State and Structural Chemistry Unit Indian Institute of Science Bangalore 560012 INDIA
On Monday 08 March 2010 01:24:17 am Pralay K Santra wrote:
Dear All,
After the final fitting in Artemis, we get the (i) total error; (ii) the final value of the parameters; (iii) the error in the parameters as well as (iv) the dependencies among the parameters. What is kind of logic used in Artemis to calculate these values? Is it Genetic optimization procedure used in this process? Can anyone help me by giving some references.
I was going through the old posts and found two posts. One is this one: http://cars9.uchicago.edu/ifeffit/FAQ/FeffitModeling and the other one mentioned in the same.
I am sorry to ask for some help which is not directly related to the XAFS.
I am not really clear how a question about error analysis could be considered as not directly related to XAFS. To my mind, error analysis is at the foundation of any scientific activity. Ifeffit uses a Levenberg-Marquardt steepest descent algorithm to find the parameters values which minimize chi-squared, which is computed in the standard fashion (Bevington's Data Reduction and Error Analysis for the Physical Sciences is my favorite text on the subject). The uncertainties are the diagonal elements of the covarience matrix, albeit scaled by the square root of reduced chi-square. The reason for this is that it is somewhere between extraordinarily difficult and impossible to fully evaluate the measurement uncertainty in an XAFS experiment. As a result, chi-square is scaled incorectly. By rescaling the diagonal elements of the covarience matrix, we are assuming that every fit is a good fit and that the only problem is the evaluation of uncertainties. Thus, if a fit is -- by some criterion -- good and is the one that you want to publish, the error bars reported by Ifeffit are 1-sigma error bars. The correlations are taken from the off-diagonal elements of the covarience matrix. Those need not be scaled and aren't. The formulas for chi-square, reduced chi-square, and the R-factor are given on pages 16 and following of this postscript file http://cars.uchicago.edu/~newville/feffit/feffit.ps My own take, for what it's worth, on all of this is explained on pages 6 to 15 of this presentation: http://xafs.org/Workshops/APS2009?action=AttachFile&do=view&target=Ravel_advanced_topics.pdf B PS: Phys. Rev. B 70, 104102 (2004) and J. Synchrotron Rad. (2005). 12, 70-74 are interesting papers about Bayesian approaches to EXAFS analysis. Ifeffit does not do Bayesian analysis. But you seem interested, so I thought I would point them out. -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/
Bruce, Some time ago I was trying to find a reference (and failed) explaining the relationship between the error bar reported by the fitting algorithm of ifeffit and one standard deviation in the fitting parameter. From your reply below it appears that they are the same ("error bars reported by ifeffit are 1 sigma"). I have read in a few old xafs papers stating, without proof, that the error bars (also obtained by the same Levenberg-Marquardt algorithm) in the fits correspond to the 95% confidence intervals which means that they are much larger than 1 sigmal, more like 3 sigma. Can you comment more on your statement about 1 sigma, or give a reference? Thank you, Anatoly ________________________________ From: ifeffit-bounces@millenia.cars.aps.anl.gov on behalf of Ravel, Bruce Sent: Mon 3/8/2010 9:11 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Logicused in Artemis to do the error minimization On Monday 08 March 2010 01:24:17 am Pralay K Santra wrote:
Dear All,
After the final fitting in Artemis, we get the (i) total error; (ii) the final value of the parameters; (iii) the error in the parameters as well as (iv) the dependencies among the parameters. What is kind of logic used in Artemis to calculate these values? Is it Genetic optimization procedure used in this process? Can anyone help me by giving some references.
I was going through the old posts and found two posts. One is this one: http://cars9.uchicago.edu/ifeffit/FAQ/FeffitModeling and the other one mentioned in the same.
I am sorry to ask for some help which is not directly related to the XAFS.
I am not really clear how a question about error analysis could be considered as not directly related to XAFS. To my mind, error analysis is at the foundation of any scientific activity. Ifeffit uses a Levenberg-Marquardt steepest descent algorithm to find the parameters values which minimize chi-squared, which is computed in the standard fashion (Bevington's Data Reduction and Error Analysis for the Physical Sciences is my favorite text on the subject). The uncertainties are the diagonal elements of the covarience matrix, albeit scaled by the square root of reduced chi-square. The reason for this is that it is somewhere between extraordinarily difficult and impossible to fully evaluate the measurement uncertainty in an XAFS experiment. As a result, chi-square is scaled incorectly. By rescaling the diagonal elements of the covarience matrix, we are assuming that every fit is a good fit and that the only problem is the evaluation of uncertainties. Thus, if a fit is -- by some criterion -- good and is the one that you want to publish, the error bars reported by Ifeffit are 1-sigma error bars. The correlations are taken from the off-diagonal elements of the covarience matrix. Those need not be scaled and aren't. The formulas for chi-square, reduced chi-square, and the R-factor are given on pages 16 and following of this postscript file http://cars.uchicago.edu/~newville/feffit/feffit.ps My own take, for what it's worth, on all of this is explained on pages 6 to 15 of this presentation: http://xafs.org/Workshops/APS2009?action=AttachFile&do=view&target=Ravel_advanced_topics.pdf B PS: Phys. Rev. B 70, 104102 (2004) and J. Synchrotron Rad. (2005). 12, 70-74 are interesting papers about Bayesian approaches to EXAFS analysis. Ifeffit does not do Bayesian analysis. But you seem interested, so I thought I would point them out. -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
I meant that error bars are related to "standard deviations of the mean", not "standard deviations"... Anatoly ________________________________ From: ifeffit-bounces@millenia.cars.aps.anl.gov on behalf of Frenkel, Anatoly Sent: Mon 3/8/2010 9:25 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Logicused in Artemis to do the error minimization Bruce, Some time ago I was trying to find a reference (and failed) explaining the relationship between the error bar reported by the fitting algorithm of ifeffit and one standard deviation in the fitting parameter. From your reply below it appears that they are the same ("error bars reported by ifeffit are 1 sigma"). I have read in a few old xafs papers stating, without proof, that the error bars (also obtained by the same Levenberg-Marquardt algorithm) in the fits correspond to the 95% confidence intervals which means that they are much larger than 1 sigmal, more like 3 sigma. Can you comment more on your statement about 1 sigma, or give a reference? Thank you, Anatoly ________________________________ From: ifeffit-bounces@millenia.cars.aps.anl.gov on behalf of Ravel, Bruce Sent: Mon 3/8/2010 9:11 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Logicused in Artemis to do the error minimization On Monday 08 March 2010 01:24:17 am Pralay K Santra wrote:
Dear All,
After the final fitting in Artemis, we get the (i) total error; (ii) the final value of the parameters; (iii) the error in the parameters as well as (iv) the dependencies among the parameters. What is kind of logic used in Artemis to calculate these values? Is it Genetic optimization procedure used in this process? Can anyone help me by giving some references.
I was going through the old posts and found two posts. One is this one: http://cars9.uchicago.edu/ifeffit/FAQ/FeffitModeling and the other one mentioned in the same.
I am sorry to ask for some help which is not directly related to the XAFS.
I am not really clear how a question about error analysis could be considered as not directly related to XAFS. To my mind, error analysis is at the foundation of any scientific activity. Ifeffit uses a Levenberg-Marquardt steepest descent algorithm to find the parameters values which minimize chi-squared, which is computed in the standard fashion (Bevington's Data Reduction and Error Analysis for the Physical Sciences is my favorite text on the subject). The uncertainties are the diagonal elements of the covarience matrix, albeit scaled by the square root of reduced chi-square. The reason for this is that it is somewhere between extraordinarily difficult and impossible to fully evaluate the measurement uncertainty in an XAFS experiment. As a result, chi-square is scaled incorectly. By rescaling the diagonal elements of the covarience matrix, we are assuming that every fit is a good fit and that the only problem is the evaluation of uncertainties. Thus, if a fit is -- by some criterion -- good and is the one that you want to publish, the error bars reported by Ifeffit are 1-sigma error bars. The correlations are taken from the off-diagonal elements of the covarience matrix. Those need not be scaled and aren't. The formulas for chi-square, reduced chi-square, and the R-factor are given on pages 16 and following of this postscript file http://cars.uchicago.edu/~newville/feffit/feffit.ps My own take, for what it's worth, on all of this is explained on pages 6 to 15 of this presentation: http://xafs.org/Workshops/APS2009?action=AttachFile&do=view&target=Ravel_advanced_topics.pdf B PS: Phys. Rev. B 70, 104102 (2004) and J. Synchrotron Rad. (2005). 12, 70-74 are interesting papers about Bayesian approaches to EXAFS analysis. Ifeffit does not do Bayesian analysis. But you seem interested, so I thought I would point them out. -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Dear Bruce,
Thanks a lot. It will help me a lot. I will go through the details of
the references which you had mentioned.
With regards,
Pralay.
On Mon, Mar 8, 2010 at 7:41 PM, Bruce Ravel
On Monday 08 March 2010 01:24:17 am Pralay K Santra wrote:
Dear All,
After the final fitting in Artemis, we get the (i) total error; (ii) the final value of the parameters; (iii) the error in the parameters as well as (iv) the dependencies among the parameters. What is kind of logic used in Artemis to calculate these values? Is it Genetic optimization procedure used in this process? Can anyone help me by giving some references.
I was going through the old posts and found two posts. One is this one: http://cars9.uchicago.edu/ifeffit/FAQ/FeffitModeling and the other one mentioned in the same.
I am sorry to ask for some help which is not directly related to the XAFS.
I am not really clear how a question about error analysis could be considered as not directly related to XAFS. To my mind, error analysis is at the foundation of any scientific activity.
Ifeffit uses a Levenberg-Marquardt steepest descent algorithm to find the parameters values which minimize chi-squared, which is computed in the standard fashion (Bevington's Data Reduction and Error Analysis for the Physical Sciences is my favorite text on the subject).
The uncertainties are the diagonal elements of the covarience matrix, albeit scaled by the square root of reduced chi-square. The reason for this is that it is somewhere between extraordinarily difficult and impossible to fully evaluate the measurement uncertainty in an XAFS experiment. As a result, chi-square is scaled incorectly. By rescaling the diagonal elements of the covarience matrix, we are assuming that every fit is a good fit and that the only problem is the evaluation of uncertainties. Thus, if a fit is -- by some criterion -- good and is the one that you want to publish, the error bars reported by Ifeffit are 1-sigma error bars.
The correlations are taken from the off-diagonal elements of the covarience matrix. Those need not be scaled and aren't.
The formulas for chi-square, reduced chi-square, and the R-factor are given on pages 16 and following of this postscript file http://cars.uchicago.edu/~newville/feffit/feffit.ps
My own take, for what it's worth, on all of this is explained on pages 6 to 15 of this presentation:
http://xafs.org/Workshops/APS2009?action=AttachFile&do=view&target=Ravel_advanced_topics.pdf
B
PS: Phys. Rev. B 70, 104102 (2004) and J. Synchrotron Rad. (2005). 12, 70-74 are interesting papers about Bayesian approaches to EXAFS analysis. Ifeffit does not do Bayesian analysis. But you seem interested, so I thought I would point them out.
--
Bruce Ravel ------------------------------------ bravel@bnl.gov
National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973
My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
-- Research Student Prof D. D. Sarma's Group (http://sscu.iisc.ernet.in/DDSarma) Solid State and Structural Chemistry Unit Indian Institute of Science Bangalore 560 012 INDIA Ph No: +91 80 2293 2336 (Extn: 2052) Fax No: +91 80 2360 1310 Mob No: +91 9845123219
participants (4)
-
Bruce Ravel -
Frenkel, Anatoly -
Pralay K Santra -
Pralay K. Santra