doubt on number of variables
Dear all, Perhaps this question is silly but I need to clarify a doubt I have: As far as I understand the number of variables refers to those parameters which have been "guessed" in the fit and it is limited by the number of independent points. If I have a group of variables but only set to guess a few of them while fixing the others, then fix some of the obtained values and guess other variables; I might include a large number of parameters in the resulting fit (even more than the independent points) without increasing the Number of variables of each individual fit. This also would keep reduced chi2 from increasing too much. Would this be cheating? The total number of parameters included is always restricted by the Number of independent points? or Can I include all the parameters I want as long as I set to guess no more that the number of independent points at the time? Thank you in advance as always, Yours Jesús esús
On 05/26/2016 09:17 AM, Jesús Eduardo Vega Castillo wrote:
As far as I understand the number of variables refers to those parameters which have been "guessed" in the fit and it is limited by the number of independent points.
Correct.
If I have a group of variables but only set to guess a few of them while fixing the others, then fix some of the obtained values and guess other variables; I might include a large number of parameters in the resulting fit (even more than the independent points) without increasing the Number of variables of each individual fit. This also would keep reduced chi2 from increasing too much.
Would this be cheating?
The answer, almost certainly, is "yes".
The total number of parameters included is always restricted by the Number of independent points?
I'm going with this one ...
or Can I include all the parameters I want as long as I set to guess no more that the number of independent points at the time?
... and I really don't like this one. The thing you seem not to be considering is that parameters have correlations. The correlations between parameters are an important part of the assessment of uncertainty. If you artificially suppress correlations between parameters in the way that you are describing, it is unlikely that the fitted results and their uncertainties would be defensible. While it is possible to probe correlations between parameters in a defensible manner by selectively setting and guessing parameters in a lengthy sequence of fits, your brief description does not give me confidence that that is what you are doing. The trick to EXAFS analysis (and any non-linear minimization analysis) is to defensibly get what information you can from the data. Rarely do we get everything we want. In those situations where we aspire to more than our data can provide, it's important to remember that something is better than nothing. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 535A Upton NY, 11973 Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/
Thanks Bruce for your answer,
Just for clarifying, this is not what I have been doing. I just was asked
to add new independent variables (individual DW factors for each path)
while I am already at the limit of independent points.
I have been using a number of parameter always lower than the number of
independent points. I have also been managing high correlations by not
varying two strongly correlated parameters at the time. But at the end I do
a final fit setting to guess all the parameters, no matter the
correlations, in order to obtain a "true" reduced Chi2 to report which
includes all parameters. Do you consider this procedure right?
2016-05-26 10:36 GMT-03:00 Bruce Ravel
On 05/26/2016 09:17 AM, Jesús Eduardo Vega Castillo wrote:
As far as I understand the number of variables refers to those parameters which have been "guessed" in the fit and it is limited by the number of independent points.
Correct.
If I have a group of variables but only set to guess a few of them while
fixing the others, then fix some of the obtained values and guess other variables; I might include a large number of parameters in the resulting fit (even more than the independent points) without increasing the Number of variables of each individual fit. This also would keep reduced chi2 from increasing too much.
Would this be cheating?
The answer, almost certainly, is "yes".
The total number of parameters included is always restricted by the
Number of independent points?
I'm going with this one ...
or
Can I include all the parameters I want as long as I set to guess no more that the number of independent points at the time?
... and I really don't like this one.
The thing you seem not to be considering is that parameters have correlations. The correlations between parameters are an important part of the assessment of uncertainty.
If you artificially suppress correlations between parameters in the way that you are describing, it is unlikely that the fitted results and their uncertainties would be defensible.
While it is possible to probe correlations between parameters in a defensible manner by selectively setting and guessing parameters in a lengthy sequence of fits, your brief description does not give me confidence that that is what you are doing.
The trick to EXAFS analysis (and any non-linear minimization analysis) is to defensibly get what information you can from the data. Rarely do we get everything we want. In those situations where we aspire to more than our data can provide, it's important to remember that something is better than nothing.
B
-- Bruce Ravel ------------------------------------ bravel@bnl.gov
National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 535A Upton NY, 11973
Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
Hi Jesus,
Yes, that procedure is fine. As long as the final, reported, fit guesses everything which is varied, whatever you do with earlier fits amounts to exploration and diagnostics.
On May 26, 2016, at 10:02 AM, Jesús Eduardo Vega Castillo
Oops, accidentally hit send before I meant to!
You might also find the information in this presentation helpful, as it addresses some of the kinds of questions you are asking: https://www-ssrl.slac.stanford.edu/conferences/workshops/srxas2011/presentat...
—Scott Calvin
Sarah Lawrence College
On May 26, 2016, at 10:23 AM, Scott Calvin
On 05/26/2016 10:02 AM, Jesús Eduardo Vega Castillo wrote:
Just for clarifying, this is not what I have been doing. I just was asked to add new independent variables (individual DW factors for each path) while I am already at the limit of independent points.
I have been using a number of parameter always lower than the number of independent points. I have also been managing high correlations by not varying two strongly correlated parameters at the time. But at the end I do a final fit setting to guess all the parameters, no matter the correlations, in order to obtain a "true" reduced Chi2 to report which includes all parameters. Do you consider this procedure right?
Well, "truth" is a difficult concept and isn't really what EXAFS analysis is all about. When we do an EXAFS analysis we are testing the extent to which data are consistent with a fitting model. The goal is to find a fitting model that is a defensible description of the data and which yields fitting results and uncertainties that are, themselves, defensible and which tell us something that we want to know about the sample which was measured. Some parameters are highly correlated and there is nothing you can do about it, regardless of how much you might want to. In a one-shell fit, S02 and coordination number are 100% correlated. It doesn't matter how much your thesis advisor wants you to get the "true" coordination number from the one-shell fit, it won't happen. So, in the end, you settle upon a fitting model and write a paper. In that paper you report on the fitting results. If the referee asks why you chose to fix a certain parameter, you better have a good reason. That's what I mean by "defensible" -- that you can explain and justify all the choices you made when fitting the data. The "procedure" is that you define a fitting model, press the big Fit button, and interpret the results. The parameters that you float, the parameters that you fix ... they are defensible when you can defend them. "Because the boss wanted two more sigma^2 parameters" is probably not a defensible argument. Here are a couple of my papers: http://dx.doi.org/10.1016/j.radphyschem.2009.05.024 http://dx.doi.org/10.1107/S1600577514014982 I refer you to these not because they are necessarily relevant to your work or even because they are especially good papers. But, in each, I discuss defensibility of fitting model in a situation where there is not nearly enough information in the measured data to properly describe the actual structure of the sample. Perhaps you'll find it helpful to see how I address this issue. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 535A Upton NY, 11973 Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/
Thank you very much Bruce and Scott,
Your answers are really helpful.
2016-05-26 11:28 GMT-03:00 Bruce Ravel
On 05/26/2016 10:02 AM, Jesús Eduardo Vega Castillo wrote:
Just for clarifying, this is not what I have been doing. I just was asked to add new independent variables (individual DW factors for each path) while I am already at the limit of independent points.
I have been using a number of parameter always lower than the number of independent points. I have also been managing high correlations by not varying two strongly correlated parameters at the time. But at the end I do a final fit setting to guess all the parameters, no matter the correlations, in order to obtain a "true" reduced Chi2 to report which includes all parameters. Do you consider this procedure right?
Well, "truth" is a difficult concept and isn't really what EXAFS analysis is all about.
When we do an EXAFS analysis we are testing the extent to which data are consistent with a fitting model. The goal is to find a fitting model that is a defensible description of the data and which yields fitting results and uncertainties that are, themselves, defensible and which tell us something that we want to know about the sample which was measured.
Some parameters are highly correlated and there is nothing you can do about it, regardless of how much you might want to. In a one-shell fit, S02 and coordination number are 100% correlated. It doesn't matter how much your thesis advisor wants you to get the "true" coordination number from the one-shell fit, it won't happen.
So, in the end, you settle upon a fitting model and write a paper. In that paper you report on the fitting results. If the referee asks why you chose to fix a certain parameter, you better have a good reason. That's what I mean by "defensible" -- that you can explain and justify all the choices you made when fitting the data.
The "procedure" is that you define a fitting model, press the big Fit button, and interpret the results. The parameters that you float, the parameters that you fix ... they are defensible when you can defend them. "Because the boss wanted two more sigma^2 parameters" is probably not a defensible argument.
Here are a couple of my papers:
http://dx.doi.org/10.1016/j.radphyschem.2009.05.024 http://dx.doi.org/10.1107/S1600577514014982
I refer you to these not because they are necessarily relevant to your work or even because they are especially good papers. But, in each, I discuss defensibility of fitting model in a situation where there is not nearly enough information in the measured data to properly describe the actual structure of the sample. Perhaps you'll find it helpful to see how I address this issue.
B
-- Bruce Ravel ------------------------------------ bravel@bnl.gov
National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 535A Upton NY, 11973
Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
participants (3)
-
Bruce Ravel -
Jesús Eduardo Vega Castillo -
Scott Calvin