[Ifeffit] Question about statistics

Olga Kashurnikova okash at mail.ru
Fri Mar 6 14:39:00 CST 2015

Hello, Dr Matt Newville,

Thank you very much for the answer.

There is a message with some corrections in the next thread - the numbers of
my hand try to check am I right about chi_square and error calculation in
IFEFFIT, were not right in the first time. I will try to attach the
calculation files later, it is not clear for me how they will be shown here.

I will write the main question and then about Bayesian analysis for thread
splitting if you don't mind. These things are interconnected that's why I
didn't split them.

The main question was about how IFEFFIT calculates chi_square and to what
minimization function it adds restraints, in case of k_space. What is the
normalization? I thought it should be
(N_idp/N_data)sum[(dat-model)^2/eps^2], and in k space only the real part of
data compared with imaginary part of model, but my hand calculation for
check this gave the different result. It is necessary for the choice of
optimal restraint (it is chi^2 without normalization that I need to compare)
and for normalization of errors for them to be covariance matrix of this
chi^2, as of Krappe and Rossner. Restraint will change errors, of course,
because covariance matrix is (Q+A)^(-1) where A is regularizer in restraint
and Q is what should be without restraint. If Q is not invertible (in case
of some parametra don't influence spectrum) it is critical, and optimal A
can be found to strictly divide the 'parametra space' and verify models.
That's why I need to rescale them well, because for bayes it should be
chi^2=sum[(dat-model)^2/eps^2]+A*sum(x-x0)**2  (A is connected with a priori
ranges of parametra, it should be not one number, but algorithm of finding
optimal diagonal A matrix is harder and I think not with IFEFFIT/Larch
help). It is the same as your (paramA-A_0)/eps_A restraint, I understood
what formula should be for a restraint, but don't understand how to weight
data and restraint part. It were Krappe and Rossner who mentioned Tikhonov
regularization if I remember right, and it is very close, if I understood
the paper about this regularization. That's what I mean in 'doesn't fit': 4
coordination spheres (Gd-O, Gd-Gd,Gd-Hf, next Gd-O for instance, and only
first is in separate peak) may be in the N_idp range, but are so correlated
that without constraints or other structure model the fit give bad results -
some parametra will leave the acceptable range etc, and I'm not sure that
without a Bayesian analysis I can define the better model of constraints,
and the sphere splitting is even more difficult to define.

So, the main question is 'how IFEFFIT calculates this, what is the formula',
because FEFFIT manual doesn't give the k-space case, and the check was
wrong. I can append calculation files in a while if needed (I need to
convert from a program to iff file), though I wrote formulae and they may be
can be verified without numerical values. 

Chi_square in your method is close to what should be minimized and for what
the covariance matrix should be found in Bayesian analysis, with the right
normalization, that's why I ask if I could use IFEFFIT to calculate this
with renormalization of the results. If I put the right normalization to
restraint, I need only rescale chi_square and errors and then calculate what
I need. It is not that I think IFEFFIT use Bayesian statistics, I understand
there are different approaches.


Thankful for your assistance,

Olga Kashurnikova, MEPhI, Moscow


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20150306/5bb62a42/attachment.html>

More information about the Ifeffit mailing list